■<3JO-H>aJOCD>r  OZO30HWS30> 


aw 

^CO 

CO 


July  1994 

Final  Contract  Report  for  Period  December  1990  •  April  1992 


NOTICES 


This  document  is  furnished  for  information  and  general  guidance  only;  it  is  not  to 
be  construed  as  a  request  for  proposal,  nor  as  a  commitment  by  the  Government 
to  issue  a  contract,  nor  as  authority  from  the  undersigned  to  incur  expenses  in 
anticipation  of  a  Government  contract,  nor  is  it  to  be  used  as  the  basis  of  a  claim 
against  the  Government.  The  furnishing  of  this  document  by  the  Government  is 
not  to  be  construed  to  obligate  your  company  to  furnish  to  the  United  States 
Government  any  experimental,  developmental,  research,  or  production  articles, 
services  or  proposals,  or  comment  with  respect  to  such  document,  the  Technical 
Objective  Document  (TOD)  program  or  any  aspects  of  either. 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any 
purpose  other  than  in  connection  with  a  definitely  Government-related 
procurement,  the  United  States  Government  incurs  no  responsibility  or  any 
obligation  whatsoever.  The  fact  that  the  Government  may  have  formulated  or  in 
anyway  supplied  the  said  drawings,  specifications,  or  other  data,  is  not  to  be 
regarded  by  implication,  or  othem/ise  in  any  manner  construed,  as  licensing,  the 
holder,  or  any  other  person  or  corporation;  or  as  conveying  any  rights  or 
permission  to  manufacture,  use,  or  sell  any  patented  invention  that  may  in  any 
way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  paper,  and  it  is  releasable  to  the 
National  Technical  Information  Service,  where  it  will  be  available  to  the  general 
public,  including  foreign  nationals. 


Project  Scientist 

Manpower  &  Personnel  Research 
Division 


Technical  Director 

Manpower  &  Personnel  Research 

Division 


WILLARD  BEAVERS,  Lt  Col,  USAF 


Chief 

Manpower  &  Personnel  Research  Division 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


PuMc  rapodlng  burOan  tor  iMi  coHadion  of  Intormailon  la  aallmatad  lo  avanpa  1  fiour  par  raaponaa.  Including  tfw  lima  for  ravlawing  loatniolona.  saarcning  axiaiing  data  aouioaa. 
galhaifng  and  makuaMng  lha  data  naadad.  and  compMing  and  ravlawing  ifw  aottaaion  of  Inlonnailon.  SacHf  commanu  ragarding  this  burdan  aatimaia  or  any  oihar  aspaa  ol  IMi 
coHadlon  of  Inlonnailon,  Including  luggaallons  lor  raducing  this  burdan.  lo  Washington  Haadquanars  Sarvloae.  Diracioraia  lor  Inlormallon  Oparatlons  and  Repons,  1 2i  S  Jallaraon 
Davis  HlohwaY.  Suite  1 204.  Arlington.  VA  22202-4302.  and  lo  lha  Otilce  ol  Manaoement  and  Budoel  PaDerwork  Reduction  Project  (0704.01881.  Washlnoton  DC  20503. _ 


1.  AGENCY  USE  ONLY  (Leane  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

_  July  1994  Final  December  1990  -  April  1992 


4.  TITLE  AND  SUBTITLE  I  5.  FUNDING  NUMBERS 


Critical  Experiments  of  HARDMAN  III  Utility  for  Use  by 
Air  Force  Analysts 


6.  AUTHOR(S) 

Russell  Flint  William  Weaver 

Paul  Rossmeissl  Sue  Dahl 
R.  Bruce  Gould 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 


HAY  Systems,  Incorporated 
2000  M  St  N.W.,  Suite  650 
Washington  DC  20036 


F49642-88-D0001 

63227F 

2922 

03 

01 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


Micro  Analysis  and  Design 
3300  Mitchell  Lane,  Suite  175 
Boulder,  CO  80301 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  AODRESS(ES) 
Armstrong  Laboratory  (AFMC) 

Human  Resources  Directorate 
Manpower  4  Personnel  Research  Division 
MPT  Integration  Branch 
7909  Lindbergh  Drive 
Brooks  AFB,  TX  78235-5352 


11.  SUPPLEMENTARY  NOTES 

Armstrong  Laboratory  Technical  Monitor:  Dr.  R.  Bruce  Gouid,  (2x0)  536-3648. 


AL/HR-CR-1 994-0002 


12a.  OtSTRIBUTlON/AVAILABIUTY  STATEMENT 


12b.  DISTRIBUTION  CODE 


Approved  for  public  release;  distribution  is  unlimited. 


13.  ABSTRACT  (Maximum  200  words) 

This  paper  documents  a  software  modification  and  validation  effort  targeted  at 
providing  a  manpower  requirements  analyses  tool  for  the  Air  Force's  Integrated  Manpower, 
Personnel,  and  Comprehensive  Training  and  Safety  (IMPACTS)  program.  Described  are 
development  and  testing  of  the  Air  Force-MANpower  CAPability  model  (AF-MANCAP)  which  was 
derived  from  the  Army's  HARDware  vs.  MANpower  (H7VRD  MAN  III)  suite  of  manpower,  personnel, 
and  training  analysis  tools.  The  paper  also  provides  results  of  field  product  reviews 
and  a  convergent  validation  test  using  the  Logistics  Composite  Model  (LCOM) . 


14.  SUBJECT  TERMS 

AF-MANCAP 

LCOM  validation 

Convergent  validation 

Manpower  requirements 

IMPACTS 

IS.  NUMBER  OF  PAGES 
116 


16.  PRICE  CODE 


OF  REPORT 
Unclassified 


NSN  ^4GD1-280-5500 


18.  SECURITY  CLASSIFICATION 

19.  SECURITY  CLASSIFICATION 

OF  THIS  PAGE 

OF  ABSTRACT 

Unclassified 

Unclassified 

Standard  Form  298  mev.  2-89) 
PiMCftwO  by  ANSI  Sid  Z3b-18 


Piwedbadby 

288-102 


Table  of  Contents 


PREFACE 


SUMMARY .  1 

I.  INTRODUCTION .  4 

Prior  Research . 4 

Origin  of  Air  Force-MANpower  CAPabilities  Analysis  Aid . 6 

Influencing  the  Weapons  System  Design . 6 

Need  for  Manpower  Analysis . 8 

II.  METHODS,  ASSUMPTIONS  AND  PROCEDURES . 9 

Overview  of  Key  Analytical  Methods . 9 

Air  Force-MANpower  CAPability  Tool  Methodology . 12 

Phase  I  -  Orientation . 15 

Phase  n  -  Database  Interfaces,  Software  Modifications,  and  Demonstrations . 16 

Phase  III  -  Convergent  Validation . 24 

III.  RESULTS . 25 

Demonstrations . 25 

Convergent  Validation . 28 

VI,  Conclusions . 29 

Demonstrations . 29 

Convergent  Validation . . 29 

Applicability  of  AF-MANCAP  to  the  Acquisition  Process . 30 

V.  RECOMMENDATIONS . 3  1 

VI.  LIST  OF  REFERENCES . 34 

VII.  GLOSSARY . 38 


ilcQS"Ston  For _ 


List  of  Figures 


Figure 

1 .  Origin  of  Air  Force-MANpower  CAPabilities  Analysis  Aid . 7 

2.  Baseline  Comparison  System  Construct  Development . 1 1 

3.  Overview  the  AF-MANCAP  Analysis  Aid . 1 3 

4.  On-Equipment  Matrix . 19 

5.  Off-Equipment  Matrix . 19 


List  of  Tables 

Table 

1 .  Key  Scenario  Variables . 1 5 

2.  Notional  New  Fighter  Database . 1 7 

3 .  Data  Element  Definition  (DED)  Used . 18 

4.  New  Fighter  Database  Restructure . 20 

5.  AF-MANCAP  Testing  Summary . 23 

6.  Convergent  Validation  Scenario  Assumptions . 24 

7.  Experimental  Design . 25 

8.  Survey  Results:  Customer  Satisfaction . 27 

9.  Convergent  Validation  Measurements . 28 

10.  Listing  of  Features  for  the  Production  Analysis  Aid . 32 


IV 


PREFACE 


This  paper  describes  die  development  and  testing  of  the  Air  Force-MANpower  CAPability 
model,  or  AF-MANCAP.  The  focus  of  this  paper  is  to  provide  a  top  level  review  of  the  AF- 
MANCAP  model.  Also,  the  paper  provides  the  results  of  a  user  survey  and  a  convergent 
validation  test.  The  convergent  validation  testing  model  was  the  Logistics  Composite  Model 
(LCOM). 

The  Army's  HARDware  vs.  MANpower  (HARDMAN  III)  suite  of  manpower,  personnel, 
and  training  analysis  tools  was  the  steppingstone  for  the  development  of  AF-MANCAP.  The 
authors  appreciate  the  support  of  Dr.  Jonathan  D.  Kaplan  and  Mr.  Rich  Maisano  from  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue, 
Alexandria  Virginia  who  provided  support  to  transition  the  Army's  manpower  analytical  concepts 
into  an  Air  Force  analytical  approach.  Also,  thanks  to  Mr.  Edward  Boyle  for  the  use  of  his  paper 
LCOM  Explained 

During  September  1991  HAY  Systems,  Inc.  demonstrated  the  AF-MANCAP  software  and 
obtained  comments  from  potential  analysts.  We  thank  the  following  personnel  who  contributed  to 
this  effort: 


Headquarters  Air  Force  Logistics  Lbmmand  participants  (3  September  1991)  -  -  Captain 
Steven  Andrasz,  Mr.  Keven  Cooksey,  Major  Greg  Grice,  Mr.  Harold  Hixson,  Mr. 
John  Madden,  Mr.  Max  Mohr,  and  Mr.  Dave  White. 

Aeronautical  Systems  Division  participants  (4  September  1991)  -  -  (Captain  Steve  Bean, 
Lieutenant  Ctolonel  Ken  Binzer,  Mr.  Billy  Bishnow,  Captain  Bob  Boeshart,  Mr.  Fred 
Conway,  Mr.  Dick  Cronk,  Lieutenant  Colonel  Paul  Cunningham,  Mr.  Don  Dyer, 
Lieutenant  Chase  McCown,  Mr.  John  Magnone,  Captain  Jack  Mohney,  Mr.  Ray 
Moore,  Mr.  Richard  Schiflfler,  Captain  Pat  Vincent,  Mr.  Allan  Wallace,  and  Lieutenant 
Cheryl  Zeek. 

Headquarters  Tactical  Air  Command  participants  (23  September  1991)  -  -  Technical 
Sergeant  Bussy  Bolster,  Major  Steve  Cooper,  Mr.  Karl  Fulnecky,  Mr.  Ed  Merry,  and 
Senior  Master  Sergeant  Jim  Schoeppel. 
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Comments  and  suggestions  to  a  draft,  received  from  several  sources,  have  improved  the 
paper's  technical  accuracy.  These  were:  Hay  Systems  Inc:  Mr.  Ken  Johnson  and  Mr.  Terry 
Garrett.  Also,  Ms.  Robin  Walthour  assisted  in  the  edit  and  document  preparation. 

This  volume  is  the  final  report  of  the  Critical  Experiment  to  modify  HARDMAN  in  software 
for  Air  Force  use.  This  effort  included  the  development  of  the  AF-MANCAP  software  and  seven 
additional  technical  reports.  The  work  was  performed  under  Contract  No  F49642-88-D0(303, 
Delivery  Order  5027. 
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SUMMARY 


The  Air  Foice-MANpower  CAPabilities  (AF-MANCAP)  analysis  aid  is  designed  to  assist  Air 
Force  acquisition  managers  in  developing  a  Baseline  Comparison  System  (BCS).  Using  AF- 
MANCAP  an  analyst  creates  a  task  level  database  from  current  operational  system  data.  The  task 
level  database  closely  represents  the  design,  operational,  and  suppon  characteristics  of  a  new 
system  under  development.  Linked  to  the  BCS  function  is  a  stochastic  simulation  process.  The 
model  supports  simulations  at  the  component  level  and  task  level  for  an  individual  system.  The 
outputs  include  reports  of  component  repair  manhours  and  associated  data.  These  data  include:  Air 
Force  Specialty  Code  (AFSC),  number  of  personnel  performing  the  task,  maintenance  task  time, 
and  mean  time  between  component  failure. 

Using  the  AF-MANCAP  model,  an  analyst  has  a  personal  computer-based  tool  to  assess  the 
impacts  of  proposed  hardware  trade-offs  for  resource  requirements  (manhour  increases  or 
decreases).  Also,  the  model  includes  the  capability  to  assess  the  impact  of  trade-offs  at  the  system 
level  in  terms  of  availability  measures.  The  availability  measures  are:  inherent,  achieved,  and 
operational. 

The  current  study  is  a  proof-of-concept  project.  The  first  phase  of  the  effort  involved  a  top- 
down  review  of  software  and  documentation  from  the  United  States  Army  HARDware  vs. 
MANpower  (HARDMAN)  project  The  changes  for  converting  the  Army’s  maintenance  and  skill 
concepts  as  well  as  types  of  activity  for  adding  Air  Force  data  were  detailed  in  a  testing  plan.  The 
second  phase  of  the  project  added  new  task  level  databases  and  covered  the  modification/debug¬ 
ging  of  the  simulation  software.  Concurrent  with  the  software  modifications,  a  structured 
demonstration  was  developed  for  the  critical  experiment  task  of  the  statement  of  work.  The 
Laboratory  Contract  Monitor  and  16  potential  users  participated  in  the  critical  experiments.  Their 
reactions  to  the  product  are  detailed  in  this  report  The  third  phase  of  the  project  implemented  a 
convergent  validation  analysis  strategy.  The  results  of  AF-MANCAP  simulation  scenarios  were 
compared  to  the  results  from  Logistics  Composite  Model  (LCOM)  simulations  using  the  same 
input  data  and  assumptions. 

The  users  participating  in  the  second  phase,  ot  critical  experiments,  were  personnel  familiar 
with  the  Air  Force's  Integrated  Manpower,  Personnel,  And  Comprehensive  Training  and  Safety 
(IMPACTS)  program.  After  viewing  the  software  and  the  available  documentation  the  personnel 
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commented  on  the  ability  of  the  product  to  fulfill  their  customer  needs.  Four  rating  levels  were 
used: 

•  Excellent  -  the  product  equals  or  exceeds  the  requirements  on  every  aspect,  and 

can  be  used  as  specified  -  -  the  product  is  very  satisfactory. 

•  Good  -  the  product  equals  or  exceeds  the  requirements  on  most  aspects.  It 

can  be  used  as  specified,  or  with  minor  limitations  —  the  product  is 
satisfactory. 

•  Fair  -  the  product  doesn't  meet  the  requirements  on  some  aspects.  It  can 

however  be  used,  but  with  major  limitations  -  -  the  product  is  almost 
satisfactory. 

•  Poor  -  the  product  does  not  meet  the  requirements.  It  cannot  be  used  -  -  the 

prcxluct  is  not  satisfactory. 

Using  the  above  rating  level,  we  assessed  customer  satisfaction  in  five  areas.  The  five  areas 
were;  functionality,  reliability,  usability,  efficiency,  and  maintainability.  The  customer  ratings,  for 
the  16  participants  were: 

•  functionaliQ'  Good  -  the  product  performs  user  required  functions,  with  only 

minor  limitation  in  some  of  them. 

•  reliability  Good  -  the  product  performs  its  intended  functions  under  ail 

conditions.  However,  infrequent  conditions  such  as  extreme  load 
out  of  specification  input  data  sets  or  function  parameter  might  result 
in  incorrect  output, 

•  usability  Good-  minimal  prior  training  is  necessary.  This  training  can  be 

done  with  a  "tutorial"  type  of  software  package,  included  in  the  main 
delivery.  On-line  itemized  "help "is  available.  Full  user  documenta¬ 
tion  is  provided. 


•  efficiency  Good  -  response  time  remains  acceptable  under  all  conditions; 

however,  performance  degradation  under  simulation  conditions  is 
noticeable.  The  software  indicated  when  products  will  be  available. 

•  maintainability  Good-  some  documentation  is  available  to  the  user. 

Since  the  1960s  the  Air  Force  has  used  the  main  frame  based  Logistics  Composite  Model 
(LCOM)  as  a  manpower  analysis  tool.  This  tool  relates  base-level  logistics  resources  with  each 
other  and  with  sortie  generating  capability.  The  four  LCOM  users  assessing  AF-MANCAP, 
during  the  critical  experiment,  quickly  recognized  that  the  personal  computer  based  AF-MANC.\P 
is  not  a  one-for-one  replacement.  Their  AF-MANCAP  pjroduct  assessment  ratings  were  lower  than 
the  12  personnel  who  were  not  LCOM  expens. 

The  third  phase,  or  evaluation  phase,  was  a  cooperative  effon  with  contractors  and 
Armstrong  Laboratory  personnel.  Hay  Systems,  Inc.  personnel  conducted  over  80  hours  of  AF- 
MANCAP  software  testing.  Armstrong  Laboratory  personnel  accomplished  the  LCOM  testing  for 
the  convergent  validation.  A  common  task  level  database  representative  of  a  notional  new  fighter 
aircraft  was  used  for  the  cr*nvergent  validation. 

Based  on  the  convergent  validarlor  effort  we  conclude  that  the  AF-MANCAP  analysis  aid 
provides  reliable  manhour  require  ncnti  .  We  thought  the  model  would  provide  manpower 
requirements;  iiowever,  ♦^his  determinaticn  could  not  be  evaluated  due  to  overall  model  size 
constraints.  This  was  a  proof-of-concept  demonstration.  We  identified  several  areas  where 
changes  will  improve  the  accuracy,  efficiency,  and  effectiveness  of  the  analysis  aid.  The  changes 
for  example;  increase  the  task  capability  from  300  to  3,000  tasks,  modify  the  sortie  sequencing 
screens,  add  task  sequencing,  and  add  life  cycle  cost  features.  With  such  enhancements,  the  AF- 
MANCAP  analysis  aid  can  support  the  early  phases  of  an  acquisition,  when  the  using  command  is 
striving  to  identify  manpower  constraints  for  preparation  of  the  Mission  Need  Statement  or  Opera¬ 
tional  Requirements  Document.  Also,  the  AF-MANCAP  analytical  approach  is  supportive  of 
activities  conducted  by  the  acquisition  activity  during  the  concept  development  phase  and  later 
acquisition  phases.  In  the  later  acquisition  phases  the  tool  could  be  used  for  trade-off  study 
evaluation. 
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AIR  FORCE-MANPOWER  CAPABILITY  ANALYSIS  AID: 
CRITICAL  EXPERIMENTS 


L  INTRODUCTION 

The  Air  Force's  new  Integrated  Manpower,  Personnel,  and  Comprehensive  Training  and 
Safety  (IMPACTS)  program  integrates,  during  the  acquisition  cycle,  six  human  systems  dis¬ 
ciplines.  The  human  systems  disciplines  are:  manpower,  personnel,  training,  safety,  health 
hazards,  and  human  factors  engineering.  The  goal  is  to  support  the  development  of  mission - 
capable  systems  that  can  be  operated,  maintained,  and  supported  effectively  at  the  lowest  life-cycle 
costs. 

The  existing  human  systems  integration  tools  and  databases  typically  provide  independent 
answers  addressing  manpower,  personnel,  and  training,  etc.  The  existing  tools,  also,  do  not  work 
in  concert  and  do  not  support  the  retention  of  historical  or  current  operational  data.  Usually,  the 
major  effort  to  analyze  the  impacts  of  the  disciplines  occurs  after  the  manufacturing  and  En¬ 
gineering  Development  phase.  In  these  situations  the  major  design  decisions  have  been  made  and 
most  life-cycle  costs  already  fixed.  One  of  the  reasons  for  the  lateness  is  the  lack  of  sufficient  data 
and  analysis  tools.  (Rossmeissl,  et  al,  1990) 

Prior  Research 

United  States  Air  Force  Research  -  -  In  support  of  IMPACTS,  several  research  effons 
arc  underway  at  the  Armstrong  Laboratory  to  develop  the  necessary  tools  and  databases  for  timely 
analysis  of  acquisition  programs.  A  front-end  analysis  tool  called  Weapons  System  Optimization 
Model  (SYSMOD)  is  under  development  This  tool  supports  the  Mission  Need  Statement  plus  the 
concept  exploration  and  definition  phases  of  the  acquisition  process.  Another  effort  at  the 
Armstrong  Laboratory  is  the  Manpower,  Personnel  and  Training  Decision  Support  System  (DSS). 
This  tool  supports  the  demonstration  and  validation  phase  plus  later  phases  of  the  acquisition 
process.  The  development  of  the  DSS  may  require  four  to  five  years  of  research  and  development. 

Once  through  development,  SYSMOD  and  the  DSS  tools  must  be  tested  and  distributed. 
Moreover,  analysts  must  be  trained.  SYSMOD  and  DSS  may  not  come  into  accepted  use  within 
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the  Air  Force  for  several  years.  Yet  the  need  for  new  tools  exists  today.  (Rossmeissl,  et  al,  1990) 

United  States  Army  Research  -  -  This  report  addresses  the  potential  of  adopting  off-the- 
shelf  technology  from  the  Army  for  the  creation  of  a  tool  for  the  immediate  use  by  the  IMPACTS 
program.  Foremost  among  the  tools  are  the  HARDware  vs.  MANpower  (HARDMAN)  tools 
developed  by  the  Army  Research  Institute  (ARI).  There  are  two  software  tools  within  the 
HARDMAN  approach.  These  tools  are  HARDMAN  11.2  and  HARDMAN  HI.  We  provide  a  brief 
overview  and  discussion  of  the  current  use  of  the  two  tools  in  the  following  paragraphs: 

•  The  HARDMAN  11.2  tool:  HARDMAN  II.2  requires  a  Vax-1 1  computer  to  host  the 
suite  of  analytical  processes.  The  HARDMAN  II.2  software  provides  an  analytical 
approach  for  early  estimation  of  manpower,  personnel,  and  training  on  Army  systems 
in  the  early  phases  of  the  acquisition  process.  The  software  was  available  for  use  after 
1985  and  has  undergone  some  degree  of  operational  testing  and  validation.  An 
upgraded  version  became  available  during  1990. 

The  cost  of  applying  the  HARDMAN  IJ.2  method  is  approximately  two  and  one- 
half  person-years  for  a  large  system,  but  varies  with  such  factors  as  system  size,  system 
complexity,  accessibility  of  data  and  experienced  analysts.  A  fairly  large  (ten  plus) 
team  of  interdisciplinary  analysts  must  conduct  an  analysis.  (Bogner,  Kibbe,  Laine, 
and  Hewitt,  1990) 

•  The  HARDMAN  III  tools.  Since  1986,  the  Army  conducted  additional  research  and 
development  for  the  creation  of  a  new  set  of  Personal  Computer  (PC)  based  tools 
known  as  HARDMAN  III.  These  tools  require  the  use  of  an  IBM  PC-AT  or  com¬ 
patible  computer.  The  software  consists  of  six  components.  Each  component  can  be 
used  either  singly  or  in  combination  for  a  determination  of  the  number,  attributes, 
availability,  and  training  needs  of  the  manpower  required  to  operate  and  maintain  the 
system.  This  methodology  is  appropriate  for  use  in  evaluating  system  designs  before 
the  decision  to  develop  a  prototype.  Also,  the  methodology  provides  trade-offs  when 
data  are  available  from  breadboard  or  prototype  hardware.  (Bogner,  Kibbe,  Laine,  and 
Hewitt,  1990) 

The  cost  of  applying  the  HARDMAN  III  methods  can  be  one-half  person-year  for  a 
large  system.  The  cost  varies  with  such  factors  as  system  size,  system  complexity, 
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accessibility  of  data  and  experienced  analysts.  The  library  of  historical  data  and  user 
friendly  software  eliminates  the  need  for  the  large  team  of  interdisciplinary  analysts. 
(Bogner,  Kibbe,  Laine,  and  Hewitt,  1990) 

Origin  of  Air  Force-MANpower  CAPabilities  Analysis  Aid 

During  1990,  Air  Force  researchers  noted  that  the  HARDMAN  HI  personal  computer  (PC) 
or  compatible  computer  features  and  user  friendly  screen  technology  had  the  potential  for 
immediate  applications  within  the  Air  Force.  Additionally,  the  Army's  manpower  evaluation 
component  of  the  software  may  be  adaptable  for  Air  Force  use.  Figure  1  shows  that  the  off-the- 
shelf  technology  was  lifted  from  the  MANpower-based  System  EVALuation  (MANSEVAL)  and 
MANpower  CAPabilities  (MANCAP)  components.  The  output  of  this  effort  was  the  Air  Force- 
MANpower  CAPabilities  (AF-MANCAP)  model. 

Influencing  the  Weapons  System  Design 

Human  resources  (manpower,  personnel  and  training)  account  for  nearly  50  percent  of  the 
yearly  operations  and  support  cost  of  weapon  systems.  Therefore,  any  policy,  design  concept,  or 
technology  that  affect  the  human  resources,  should  be  evaluated  for  trade-offs  and  appropriate 
input  made  to  the  acquisition  decision  process.  AF-MANCAP  supports  trade-off  analyses  at  the 
task  level  and  serves  two  key  purposes:  influence  design(s)  and  provide  information. 

First,  an  AF-MANCAP  analysis  aid  can  identify  and  limit  requirements  for  manpower, 
particularly  for  tasks  that  demand  support  from  more  than  one  individual.  Also,  the  analysis  aid 
tracks  support  needs  for  different  Air  Force  Specialties  (AFS).  The  focus  of  this  analysis  is  to 
irfiuence  design.  Design  influence  trade-off  studies  can  identify  high  driver  tasks  and  thereby  help 
to  specify  mission  need  goals  or  constraints. 


6 


SPARC:  Operations  &  Maintenance  Criteria 


Second,  an  AF-MANCAP  analysis  aid  supports  work  force  planning.  Planning  in  this 
context  addresses:  the  structure  of  the  APS  and  the  number'^  of  personnel.  The  focus  of  this 
analysis  is  to  provide  information  about  the  work  force.  He. -ever,  trade-off  studies  of  the  type 
supported  by  AF-MANCAP  cannot  be  easily  quantified  in  isolation  from  other  factors.  In  this 
instance,  AF-MANCAP  provides  information  on  work  force  requirements  with  the  overall 
requirements  evaluated  against  system  level  measures  of  availability  (e.g.,  inherent,  achieved, 
operational,  and  sortie). 


Need  for  Manpower  Analysis 

According  to  Boyle  (1991)  "requirements  documents  from  users  (e.g.,  statement  of  need, 
system  operational  requirements  document)  will  often  state  manpower  targets  for  direct  main¬ 
tenance  manhours  per  operating  hour  or  manpower  spaces  per  unit  In  practice,  this  is  typically  all 
that  is  meant  by  design  influence  for  human  resources.  In  effect,  manpower,  personnel  and 
training  (MPT)  analysis  means  manpower  analysis." 

In  the  late  1960s  early  1970s  the  Air  Force  demonstrated  the  need  for  responsive  methods  for 
predicting  maintenance  manpower  requirements  during  weapons  system  development  (Tetmeyer 
and  Moody  1974).  Evolving  from  this  early  research  was  a  maintenance  manpower  simulation 
named  the  Logistics  Composite  Model  (LCOM).  Through  simulation,  the  impact  of  availability  of 
all  types  of  support  resources  on  the  operational  status  can  be  assessed.  This  approach  was 
applied  to  the  A- 10  weapon  system,  using  early  estimates  of  maintenance  task  data  for  the  new 
aircraft  coupled  with  detailed  operational  scenarios  and  maintenance  concepts.  The  A- 10  model 
reflected  Air  Force  experience  with  comparable  subsystems  and  equipment  on  existing  aircraft, 
factored  for  the  new  design  and  environment  (Tetmeyer  and  Moody  1974). 

The  early  LCOM  efforts  grew  into  a  composite  of  individual  programs  written  in 
SIMSCRIPT  n.5.  These  communicate  directly  with  each  other  and  function  as  a  unit.  The  LCOM 
software  provides  simulation  support  to  studies  concerning  Air  Force  base  level  functions,  e.g., 
operations,  maintenance,  and  supply.  The  LCOM  software,  together  with  the  data  representing  a 
weapons  system's  environment,  form  either  peacetime  or  wartime  base  level  study  models.  The 
model  permits  the  analysis  of  weapons  system  support  requirements  (Dengler  1981). 

LCOM  modeling  support  is  often  extensive  and  the  development  time  lengthy.  Past  model 
development  for  the  validation  of  Major  Command  manpower  requirements  can  consume  six  or 
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more  person-months  of  analytical  support.  The  activities  includt,;  ollection  of  the  maintenance 
task  data,  validation  of  the  data,  development  of  the  operational  S'  enario,  creation/testing  of  the 
simulation  networks  and  report  production.  Modeling  of  thi  r  lagnitude  during  new  weapon 
system  development  is  usually  accomplished  during  the  later  str.  ge  >  of  the  acquisition  process. 

Some  reduction  in  the  LCOM  modeling  support  requirements  and  development  times  are 
possible  through  top  level  modeling  of  existing  networks.  Analysts  from  the  Directorate  of 
Manpower,  Personnel,  and  Training,  Aeronautical  Systems  I  )i  vision  (ASD/ALH),  in  report  some 
success  in  providing  early  estimates  support.  Finished  r  ;ports  were  produced  in  weeks,  and 
supported  decisions  in  the  early  stages  of  the  acquisition  cy  le 

AF-MANCAP  does  not  replace  the  LCOM  model.  •  Ic  wever,  the  AF-MANCAP  analysis  aid, 
with  recommended  refinements,  can  satisfy  a  requiremer.  f  jr  a  PC-based  comparability  manpower 
tool.i 


Section  II  of  this  report  focuses  on  the  methods,  issumptions  and  procedures  for  the  conver¬ 
sion  of  the  manpower  evaluation  components  of  HAIlE-MAN  HI  for  Air  Force  use.  Included  is  a 
structured  interview  process  for  soliciting  potential  users'  reactions  to  the  AF-MANCAP  software. 
Lastly,  this  section  includes  the  critical  experiment  -  comparison  of  AF-MANCAP  results  to 
LCOM.  Section  IE  addresses  the  results  of  the  potential  usct  reactions  and  the  critical  experiment 
Sections  IV  and  V  of  this  report  address  the  conclusions  and  recommendations. 

SECTION  n.  METHODS,  ASSUMPTIONS  AND  PROCEDURES 
Overview  of  Key  Analytical  Methods 

The  AF-MANCAP  analysis  aid  applies  two  key  analytical  methods.  The  methods  are 
baseline  comparison  analysis  and  simulation  modeling.  Some  background  information  associated 
with  each  analytical  method  is  provided  to  acquaint  the  layman  reader  with  the  applications  within 
AF-MANCAP. 


1  The  requirement  for  a  PC  based  tr  anpower  tool  is  documented  in  the  AL/HRMM 
Manpower,  Personnel,  and  Training  Neeos  Statement  reference  material.  Headquarters  Military 
Airlift  Command  provided  the  need  statement  in  1989. 


9 


Baseline  Comparison  System  C  onstruct  Development  -  -AF-MANCAP  is  based 
on  the  comparability  analysis  proi  ess.  This  process  derives  systematic  estimates  of  the  human 
resource  requirements  of  emerging  eapon  systems  by  extrapolating  from  the  known  requirements 
of  similar  operational  systems  and  subsystems.  The  mission  need  determination  phase  of  an 
acquisition  program  ends  in  the  Air  Force's  assessment  of  its  mission  need(s).  The  needs  are 
documented  in  the  Mission  Need  Statement  (MNS).  (DoDD  5000.1)  If  a  need  for  a  new  system 
emerges  from  the  process,  it  results  ’tom  validated  deficienci''".  in  the  predecessor  system,  a 
system  currently  in  the  inventory.  By  definition,  the  predecessor  system  is  unable  to  satisfy  the 
functional  requirements  of  the  new  syst-  m  However,  the  functional  requirements  information  in 
the  MNS  usually  focuses  on  predecesst  r  deficiencies.  Missing  from  the  MNS  is  a  full  set  of 
functional  requirements  associated  with  tk:;  new  system. 

The  process  of  identification  of  the  fi  nctional  requirements  (e.g.,  two  crew  members,  and 
short  field  landing  capability)  and  mapping  ti  ose  requirements  to  specific  equipment  configurations 
is  a  construct  process.  This  process  is  facilitated  by  the  AF-MANCAP  tool.  In  theory,  the  AF- 
MANCAP  library  would  contain  construct  data  from  operational  systems  and  subsystems.  Based 
on  a  full  set  of  functional  requirements,  the  t  lalysts  would  link  the  functional  requirements  to 
specific  equipment  configurations  from  prede  essor  systems  or  subsystem.  In  this  process,  the 
analyst  identifies  the  functional  requirements  -  *  lowing  what  the  system  must  do  -  and  links  these 
to  equipment  configurations  -  the  hardware  that  will  perform  the  mission.  Ideally,  the  identified 
components  should  meet  the  design,  operatioi  al,  and  support  needs  implicit  in  the  overall 
functional  requirements.  The  system  level  construct  created  from  this  process  is  called  a  Baseline 
Comparison  System  (BCS). 

As  defined  in  MIL-STD  1388-1 A  (March  1991 )  the  BCS  is  "a  current  operational  system,  or 
a  composite  of  current  operational  subsystems,  i  hich  most  closely  represents  the  design, 
operational,  and  support  characteristics  of  the  new  system  under  development."  Components  of 
the  BCS  may  be  drawn  from  the  predecessor  system  ai.d  other  comparable  existing  systems  in  the 
Department  of  Defense  or  the  North  Atlantic  Treaty  Organization  inventory.  The  BCS  should 
closely  approximate  the  design,  operational,  and  support  characteristics.  This  concept  is  illustrated 
in  the  AF-MANCAP  demonstration  library.  The  library  contains  the  representative  subsystems 
that  may  satisfy  a  notional  new  tactical  fighter  requirement.  Individual  data  entries  in  the  BCS 
came  from  data  files  associated  with  current  predecessor  systems.  Examples  of  the  data  are; 
landing  gear  of  the  F-15,  radar  of  the  F-16,  and  flight  control  system  of  the  F-18. 
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Figure  2  illustrates  the  BCS  process.  To  qualify  for  inclusion  in  the  BCS,  a  candidate 
component  must  have  mature  data  available.  Such  data  are  used  to  demonstrate  the  likely  human 
resource  considerations  under  fielded  conditions.  The  data  are  found  in  the  Core  Automated 
Maintenance  System  (CAMS)  or  Logistics  Composite  Model  networks.  Information  can  also  be 
obtained  from  subject  matter  experts  and  from  the  maintenance  tasks  described  in  the  predecessor 
technical  orders. 


System  _ 
Functions 


Mission 
Need  ~ 
Statement 


^  Functional _ 

Criterion? 

Operational 
&  Support  Needs? 

♦ 


Baseline 

Comparison 

System 

Candidate 


Selection  from  _ 

Predecessor  Library 

_ ^  Selection  from 

Supplemental  Data 


Source:  DqieRmentof  Amy:  HARDMAN  compiMiy  Mielyiis  mahadology  guide  (Volume  D 
Alexendiia,  VA:  Amy  Refcarth  Imtinae.  (Monied  for  DoD  SOOO.I  changes) 


Figure  2.  Baseline  Comparison  System  Construct  Development 


Proposed  System  Construct  Development  -  -  Another  feature  of  AF-MANCAP  is  the 
capability  to  create  a  construct  of  the  Proposed  System  (PS).  The  data  associated  with  the  PS  is 
defined  as  less  technologically  mature.  As  such,  the  PS  can  include  data  from  test  or  engineering 
estimates.  Or  the  PS  contains  data  from  individual  contractor  designs  (e.g.,  engineering 
estimates).  Typically,  such  data  are  found  in  the  Logistic  Suppon  Analysis  Record  (LSAR). 
Differences  between  the  BCS  and  PS  can  be  analyzed  with  the  AF-MANCAP  analysis  aid. 


Modeling  Database  Content  and  Simulation  •  -  The  main  function  of  the  BCS, 
described  above,  is  to  create  a  database  that  reflects  the  rate  of  scheduled  and  unscheduled 
maintenance.  Included  are  supporting  data:  Air  Force  Specialty  Code,  task  time,  task  frequency, 
and  crew  size.  The  BCS  database  should  be  developed  according  to  the  work  unit  code  structure 
(TO  00-20-2).  Each  3-digit  work  unit  code  that  satisfies  a  functional  requirement  is  shown.  The  3- 
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digit  level  is  recommended  because  the  troubleshooting,  remove  and  replace,  inspect,  or  adjust 
maintenance  tasks  are  normally  reported  at  this  level.  Within  each  system,  significant  line 
replaceable  units  at  the  4-  or  5-  digit  work  unit  code  level  should  be  identified,  if  at  all  possible. 
However,  this  level  of  information  is  not  normally  available.  Very  early  analysis  is  limited  to  the  3- 
digit  level  of  detail.2 

Simulation  nK)deling  is  applied  within  AF-MANCAP  to  address  variables  such  as  unit  size  or 
availability  of  different  specialists.  Simulation  modeling  is  used  to  answer  complex  questions 
about  the  manning  and  mix  to  support  a  new  system  under  a  range  of  different  operational 
conditions.  The  simulation  approach  adopted  under  AF-MANCAP  was  to  allow  the  user  the 
freedom  to  sequence  up  to  five  events  over  a  24-hour  period.  The  event  sequencing  features  of  AF- 
MANCAP  can  address  the  duration  of  a  some  and  the  frequency  of  a  turnaround.  Compared  to  the 
LCOM  networking  simulation  features,  the  AF-MANCAP  networking  simulation  features  are 
simplistic.  As  an  example,  AF-MANCAP  currently  does  not  allow  identification  of  the  takeoff 
pattern  and  sequencing  of  tasks  by  dependencies. 

The  AF-MANCAP  is  a  simplistic  approach  using  PC  hardware  and  operating  systems.  The 
software  design  has  some  cuirent  limits,  e.g.,  only  300  tasks  can  be  held  in  memory  during  a 
simulation.  While  some  proponents  of  lai^e-scaJe  simulation  development  may  argue  that  this  is  a 
weakness  of  AF-MANCAP,  it  can  be  a  strength.  Complex  scenarios  and  detailed  databases  are  not 
normally  available  early  in  the  requirements  development  phase  of  an  acquisition.  Therefore,  the 
strength  of  AF-MANCAP  rests  in  the  simplistic  approach  to  scenario  and  database  development. 
For  example,  the  entire  scenario  development  process  for  AF-MANCAP  takes  approximately  15 
minutes. 


Air  Force-RI  \Npower  CAPability  Tool  Methodology 

The  Air  Force-MANpower  CAPability  (AF-MANCAP)  tool  is  a  stochastic  probability  process. 
TThe  process  meets  the  definition  of  a  strong  stationary  stochastic  process  (Pritsker  and  Pedgen 
1990)  or  a  set  of  ordered  random  variables. 


2  AF-MANCAP  supports  alternative  database  file  arrangements.  We  used  the  Logistic 
Support  Analysis  control  number;  however,  any  task  description  can  be  used.  The  current  file 
allows  up  to  50  alphanumeric  characters  for  the  task  description. 
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AF-MANCAP  features  user-friendly  inputs,  use  of  a  personal  computer,  and  standard 
screens.  An  analyst  develops  scenarios  that  are  sufficiently  simple  that  significant  amounts  of  user 
training  are  not  needed  for  modeling.  The  model  invokes  three  major  activities:  development  of  a 
baseline  comparison  system,  simulation  of  one  system,  and  simulations  at  various  levels.  The 
analytical  efforts  address  the  same  model,  design,  and  series  system.  Refer  to  Figure  3  for  an 
overview  of  the  AF-MANCAP  analysis  aid. 

First.  ActivitjL-  Baseline  Comparison  System  (BCS)  Development  -  -  For  this 
activity,  data  come  from  task  level  databases  within  the  software.  The  task  level  databases  are 
called  the  library.  The  library  includes  task  level  information  on  Army  systems  such  as  helicop¬ 
ters,  tanks,  and  vehicles.  Under  this  contract  a  demonstration  notional  new  fighter  task  level 
database  was  added  to  the  existing  library  files.  The  library  contains  the  maintenance  parameters  of 
each  compionent.  This  information  includes: 


a  task  description  (this  is  user  detinable  within  a  range  of  50  alpha/numeric  characters. 
For  this  development  a  Logistic  Support  Analysis  Record  format  was  used); 


Simulalioni  al  various  lavds 
unit,  wtn(,  or  Air  Force”"- 


PURPOSE 

•  Systematic  estimates  of  the 
human  resource  requirements 

•  Development  of  Baseline 
Comparison  Systems 

•  Trade-off  studies 


Model  Output 

•  Manpower 

•  System  R&M.,— -  . . 


Figure  3.  Overview  the  AF-MANCAP  analysis  aid. 
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how  often  the  component  needs  maintenance; 


•  what  type  of  maintenance  is  needed  (trocbleshoot,  remove  &.  replace,  inspect,  adjust  & 
repair,  or  test  &  check); 

•  what  category  of  maintenance  is  needed  (scheduled  or  unscheduled); 

•  what  level  of  maintenance  is  associated  with  the  maintenance  (on-equipment,  off- 
equipment,  or  depot  level); 

•  who  will  do  the  maintenance  (the  specific  AFSC)  and  the  number  of  personnel 
associated  with  the  maintenance  (team  size); 

=  1.V/W  long  the  niaintenance  action  will  take;  and 

•  whether  the  need  for  this  maintenance  will  interrupt  the  current  mission. 

Using  the  BCS  model  features  allows  the  analyst  to  use  the  library  database  and  select  or 
rearrange  the  tasks  between  system  and  subsystem  level.  Also,  the  BCS  feature  allows  for 
refinement  of  the  task  data  or  the  creation  of  new  task  level  databases.  The  new  BCS  can  be 
named,  saved,  or  exported  to  other  users.  After  development  the  BCS  is  automatically  linked  to 
the  simulation  activities  described  below.3 

Second  Activity  -  Simulation  of  one  system  -  •  For  this  activity,  stochastic  network 
simulations  are  run  at  the  component  level  or  task  level.  Only  one  system  operates.  The  simula¬ 
tion  generates  a  table  of  component  and  subsystem  failure  probabilities  plus  associated  data  per 
component  for  one  system.  These  data  include  the  Air  Force  Specialty  Code  (AFSC),  number  of 
personnel  performing  the  task,  maintenance  task  time,  and  mean  operational  time  between 
component  failure.  Scenario  development  and  model  execution  follow  the  data  entry.  A  standard 
series  of  reports  is  retrievable  to  the  screen,  and  options  are  available  for  printing  output  reports.  A 
key  report  example  is  manhour  predictions.  The  reports  include  the  identification  of  the  tasks 
associated  with  each  AFSC  and  the  manhours  for  tasks  at  various  maintenance  levels.  The 

3  Currently,  only  300  task  descriptions  can  be  retrieved  from  a  library  for  BCS  modeling. 
This  limit  is  a  constraint  of  the  disk  operating  system  (DOS)  and  hardware.  Recommended 
conversion  of  AF-MANCAP  to  WINDOWS  will  remove  the  BCS  constraint 
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maintenance  levels  are:  on-equipment,  off-equipment,  or  depot  maintenance.  The  interface 
between  this  activity  and  the  next  activity  is  simplified  with  an  automatic  transfer  of  data. 

Third  Activity  -  ■■Simulation  for  several  systems  of  the  same  model,  design. 
and  series  -  -  For  the  third  activity,  the  above  components  and  associated  task  level  data  support 
the  analysis  of  support  requirements  for  a  range  of  systems  of  the  same  model,  design,  and  series. 
Under  the  first  function  of  this  activity,  the  user  develops  a  scenario  for  major  activities  over  a  24 
hour  period.  For  example,  the  analyst  enters  an  aircraft  sortie  generating  sequence  of  maintenance, 
alert,  fly,  and  recovery.  This  simple  sequencing  of  events  creates  the  stochastic  network. 
Component  failures  occur  as  sorties  accumulate  during  the  flying  sequence.  After  the  flight 
sequence,  the  available  maintenance  resources  are  committed  for  the  duration  of  the  repair  time  for 
the  failed  component.  Repair  times  and  maintenance  actions  are  accumulated  by  AFSC  and 
component  over  the  duration  of  the  simulation.  The  second  major  function  of  this  activity  is  the 
input  of  key  scenario  variables.  The  variables  and  associated  ranges  are  shown  in  Table  1  below. 

Table  1.  KEY  SCENARIO  VARIABLES 

_ Variables _ Ranee _ 

Duration  1  to  180  days 

Systems  1  to  500  systems 

Activity  levels  Number  rounds  fired,  sorties  flown,  and  operating  hours 

AFSC  Constraints  1  to  999 


A  standard  series  of  reports  is  retrievable  to  the  screen  and  options  are  available  for  printing 
output  reports.  Readers  may  refer  to  Appendix  A  for  additional  AF-MANCAP  details. 

The  overall  research  effort  was  driven  by  the  statement  of  work  that  specified  three  phases  for 
the  project.  These  were:  Phase  I  -  orientation;  Phase  II  -  database  interfaces,  software 
modifications,  and  demonstrations;  Phase  HI  -  convergent  validation.  A  summary  is  provided  for 
each  phase  of  the  project. 

Phase  I  •  Orientation 

The  general  strategy  for  developing  AF-MANCAP  focused  first  on  understanding  the  Army's 
approach  to  the  database  design  and  the  simulation  network.  During  January  and  February  1991, 
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the  research  team  collected  information  about  the  Army's  MAintenance  MANpower  (MAMA) 
analysis  aid  and  MANpower  CAPabilities,  second  version  (MANCAP II)  analysis  aid.  The  Anny 
Research  Institute  provided  software  demonstraticiis,  documentation,  and  supported  two  meetings. 
Based  on  the  information  gathered  and  demonstrations  of  the  HARDMAN  m  products,  a  plan  of 
attack  was  developed  for  software  modification.^  In  March  of  1991,  the  following  software 
modification  were  approved  for  prototype  testing  (Flint  and  Rossmeissl,  1991): 

Add  Air  Force  symbols  to  the  start  up  screen  and  change  the  product  name  from 
MANCAP  n  to  AF-MANCAP. 

•  Convert  the  Army’s  Military  Occupational  Specialty  (MOS)  to  Air  Force  Specialty 
Code  (AFSC)  and  increase  the  characters  in  the  AFSC  field  firom  five  to  six. 

•  Convert  the  Army's  maintenance  level  terms  (Organization,  Direct  Support,  General 
Support,  and  Contact  Teams)  to  Air  Force  maintenance  terms  (On-equipment,  Off- 
equipment,  Depot,  and  Aircraft  Battle  Damage  Repair). 

•  Modify  the  mobility  equipment  group  from  the  term  "miles"  to  read  "miles/sorties." 

•  Change  the  terms"  preventative"  maintenance  to  read  "scheduled"  maintenance,  and 
change  the  term  "corrective"  maintenance  to  read  "unscheduled"  maintenance. 

The  above  changes  were  required  before  demonstration  of  the  product  to  Air  Force  analysts. 
Other  product  improvements  were  noted  during  the  Phase  I  orientation  but  were  deferred  for  later 
implementation.  For  example,  the  Army  software  tools  accept  only  300  maintenance  tasks  for 
simulations.  Under  typical  conditions,  the  Air  Force  may  require  over  3,000  maintenance  tasks. 
A  list  of  the  recommended  changes  may  be  found  in  Section  V  of  this  report. 

Phase  II  -  Database  interfaces,  software  modifications,  and  demonstrations 

Database  Interfaces  -  ‘We  used  a  maintenance  task  database  representative  of  a  notional 
new  fighter.  This  database  supports  other  research  at  Armstrong  Laboratory  (Boyle,  Plassenthal, 

4  The  contract  effort  also  included  a  review  of  the  PERSEVAL  tool  for  potential  Air  Force 
utilization.  During  the  Phase  I  effort  we  found  that  the  PERSEVAL  taxon  structure  was  not 
adaptable  to  the  Air  Force's  classification  structure. 


Weaver,  1991).  The  notional  new  fighter  database  includes  subsystems  from  existing  fighter 
aircraft  arranged  by  work  unit  code.  The  maintenance  task  database  was  used  for  experiment 
design.  We  used  known  data  to  compare  the  AF-MANCAP  results  to  LCOM  results.  Also,  the 
fighter  database  provided  information  to  build  a  Logistic  Support  Analysis  Record  (LSAR).  We 
used  the  LSAR  format  to  show  a  proposed  system  construct  capability.  The  overall  contents  of  the 
notional  new  fighter  task  database  are  summarized  in  Table  2. 


Table  2.  NOTIONAL  NEW  FIGHTER  DATABASE 


Work  Unit  Code  Subsystem _ Work  Unit  Code _ Subsystem 


11  Airframe 

F-16,  F-15,  F/A-18 

49  Misc  Utilities 

F/A-18 

12  Cockpit 

F-16,  F-15,  F/A-18 

51  Instruments 

F/A-18 

13  Landing  Gear 

F-15 

62  VHFi 

F/A-18 

14  Flight  Controls 

F-16,  F-15,  F/A-18 

63  UHF2 

F/A-18 

24  Auxiliary  Power 

F-16,  F-15,  F/A-18 

64  Interphone 

F/A-18 

27  Propulsion 

F-15 

65  IFF3 

F/A-18 

29  Power  Plant 

F/A-18 

66  Radio  Beacon 

F/A-18 

41  Environmental 

F-16 

67  Comm.  Nav/IFF 

F/A-18 

42  Electrical 

F-15 

71  Radio  Nav. 

F/A-18 

44  Lighting 

F-15 

72  Radar/Bomb  Nav. 

F/A-18 

45  Hydraulics 

F/A-18 

74  Fire  Control 

F/A-18.  F-15 

46  Fuel 

F-15 

75  Weapons 

F/A-18 

47  Oxygen 

F-16 

76  ECM4 

F/A-18 

1.  Very  High  Frequency  (Radio) 

2.  Ultra  High  Frequency  (Radio) 

3.  Identification  Friend  or  Foe 

4.  Electronic  (Counter  measures 


We  modified  the  task  nomenclature  associated  with  the  maintainability  and  reliability  data 
before  uploading  in  AF-MANCAP.  The  first  position  of  the  new  task  database  included  the 
identification  of  Logistic  (Hontrol  Numbers  associated  with  a  typical  LSAR.  The  data  associated 
with  the  rest  of  the  fields  may  be  found  in  Table  3  below.  The  Table  3  database  was  prepared  in 
American  Standards  for  Communications  Information  Interchange  (ASCII)  format.  Wherever 
possible  we  linked  AF-MANCAP  data  fields  with  data  representative  of  the  LSAR  file. 
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Table  3.  DATA  ELEMENT  DEFINITION  (DED)  USED 


AF-MANCAP 

Title _ ! _ Position _ 

Field 

Tvoe 

DED 

Number 

DED 

Name 

Task 

1 

50a/n 

197 

Logistic  Control  Number 

467 

Task  Code 

181 

Item  Name 

Comptype 

2 

In 

n/a 

See  note^ 

Organization  Code 

3 

In 

n/a 

See  note5 

Corrective  Code 

4 

In 

n/a 

See  note6 

AFSCl' 

5 

6a/n 

316 

Person  Identifier 

Nr  AFSC  1 

6 

3n 

269 

Nr.  of  Persons  Per  Skill  Specialty  Code 

AFSC2 

7 

6/an 

n/a 

Not  provided  by  LSAR 

Nr  AFSC  2 

8 

7n 

n/a 

Not  provided  by  LSAR 

Unit  Metric 

9 

244 

Measurement  Base 

MOUBF2 

10 

236 

Mean  Time  Between  Maint  Actions 

MTTR3 

11 

222 

Mean  Man-Hours  Per  Skill  Specialty 

1 .  Air  Force  Specialty  Code 

2.  Mean  Operational  Units  Between  Failure 

3.  Mean  Time  To  Repair 

4.  AF-MANCAP  task  code  developed  from  1st  position  of  the  LSAR  task 
code 

5 .  AF-MANCAP  organization  code  developed  from  3rd  position  of  the  LSAR 
task  code 

6 .  AF-MANCAP  Corrective  or  Preventative  Maintenance  code  developed  from 
1st  and  2nd  position  of  the  LSAR  task  code 


During  March  of  1991,  the  preliminary  integrated  AF-MANCAP  software  was  delivered  by 
Micro  Analysis  and  Design,  Inc.  This  AF-MANCAP  software  and  its  database  management 
system  (R:BASE  System  V)  provides  the  technology  for  the  transfer  of  ASCII  format  task 
database  into  AF-MANCAP.  The  FileGateway  options  of  the  R.BASE  System  V  software  allow 
conversion  of  data  from  all  the  most  popular  software  formats  (Taylor  87). 


We  could  not  make  a  direct  FileGateway  transfer  of  the  notional  new  fighter  task  database 
into  the  AF-MANCAP  file  structure.  First,  wc  created  mapping  methods  to  conven  the  Logistic 
Support  Analysis  (LSA)  task  function  codes  into  one  of  five  possible  AF-MANCAP  maintenance 
codes.  Examples  of  the  AF-MANCAP  codes  are:  inspect,  tcst/check,  adjust/repair, 
remove/replace,  and  troubleshoot.  The  key  mapping  assumptions  came  from  a  program  that 
extracts  Logistic  Support  Analysis  Records  data  and  reformats  it  into  LCOM  input  data  (Cronk 
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1989).  The  reformatting  procedures  were  mapped  to  equivalent  AF-MANCAP  code.  Since  LSA 
task  function  codes  vary  between  on-equipment  and  off-equipment  tasks,  we  developed  unique 
conversion  tables  for  on-equipment  and  off-equipment.  The  results  of  this  effort  are  shown  in 
Figures  4  and  5. 


Figure  4.  On-equipment  matrix. 


Figures.  Off-equipment  matrix. 
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Our  next  database  preparation  action  covered  the  subdivision  of  the  1,655  notional  new 
fighter  tasks  descriptions.  We  decomposed  most  of  the  library  files  into  groups  of  300  or  less 
tasks.  This  was  done  to  work  within  the  300  tasks  limits  of  AF-MANCAP.  The  results  of  this 
effort  are  shown  in  Table  4. 


Table  4.  NEW  FIGHTER  DATABASE  RESTRUCTURE 


Number  of 
Subsystems 

Number  of 
Tasks 

F-15  On-Equipment 

9 

165 

F-16  On-Equipment 

9 

204 

F-18  On-Equipment 

19 

4271 

Engine  On-Equipment 

1 

74 

Armament  On-Equipment 

1 

34 

F-15  Off-Equipment 

7 

129 

F-16  Off-Equipment 

8 

68 

F-18  Off-Equipment 

18 

283 

Engine  Off-equipment 

2 

190 

Armament  Off-^uipment 

1 

30 

Support  General 

1 

Total  1655 

1 ,  Currently  only  300  tasks  can  be  retrieved  firom  a  library  for  modeling. 


Software  Modifications  •  •  Once  we  entered  the  notional  new  fighter  aircraft  databases 
into  the  AF-MANCAP  library  function,  we  started  preliminary  testing.  Also,  we  changed  the 
software  to  improve  the  model  performance.  As  an  example,  initially  the  model  would  abort 
simulation  scenarios  of  180  days.  In  our  final  version  of  the  software,  the  model  run  time  for  the 
18u  day  scenario  was  72  minutes.  We  identified  the  need  for  product  changes  from  April  through 
December  1991.  Appendix  E  contains  a  summary  of  the  software  development  effort. 


We  conducted  over  80  hours  of  simulations  and  34  tests  of  the  AF-MANCAP  software.  We 
tested  AF-MANCAP  in  unconstrained  and  the  constrained  mode.  The  results  are  summarized  in 
Table  5. 

Demonstrations  -  -  Concurrent  with  the  software  testing,  we  developed  an  AF-MAN¬ 
CAP  demonstration  using  the  notional  new  fighter  aircraft  database.  We  used  a  briefing  format  to 
introduce  potential  users  to  the  AF-MANCAP  product.  This  required  about  one-half  hour  with 
questions  and  answers.  Following  the  briefing  potential  users  saw  all  of  the  AF-MANCAP 
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features.  We  p^'ovided  a  step-by-step  viewing  of  the  AF-MANCAP  computer  generated  images. 
A  high-resolution  liquid  crystal  display  was  placed  on  top  of  an  overhead  projector.  This  required 
about  one  and  one  half  hours  with  questions  and  answers. 

A  standard  presentation  scenario  was  used  for  each  demonstration  (Flint  and  Dahl,  August 
1991).  The  demonstration  scenario  was  approved  by  the  Laboratory  Contract  Monitor  during  July 
1991.  During  August  1991,  we  made  two  presentations  at  Wright-Patterson  Air  Force  Base, 
Ohio,  and  one  presentation  at  Langley  Air  Force  Base,  Virginia. 

We  developed  a  three  part  survey:  Part  I  -  collected  background  information  about  the 
participant.  Part  II  -  contained  questions  to  assess  if  the  software  products  fulfill  the  potential  user 
needs,  and  Part  III  -  contained  questions  about  requirements  for  a  production  version  of  the 
software.  With  each  presentation,  potential  users  were  provided  the  survey  questions  to  capture 
user  reactions  to  the  analytical  tools  and  the  manpower  analysis  process. 

Pan  n  of  the  survey  addr-^sses  user  system  interface  issues.  We  developed  the  questions 
after  an  extensive  literature  survey  and  interviews  with  software  standards  experts  at  Headquarters, 
Air  Force  Systems  Command.  We  eliminated  methods  of  assessing  the  AF-MANCAP  software 
that  address  features  such  as  design  consistency,  design  flexibility,  and  design  context  (Goodwin 
1988,  Shneiderman  1987,  and  Boehm  1989).  According  to  Goodwin  under  design  context  "the 
user  should  always  know  how  they  reached  a  certain  point,  what  data  are  being  display,  the  date 
and  time  of  data  updates,  and  other  contextual  information  that  will  help  them  accomplish  their 
task."  It  was  our  conclusion  that  the  software  demonstrated  such  design  features.  Also,  the  above 
methods  applied  to  the  development  of  software  or  /etlLution  of  a  software  product  and  not  to  the 
assessment  of  a  product  once  it  is  realized. 

The  International  Standards  Organization  guide  (ISO  1991)  provides  metrics  for  software 
quality  measurements,  ratings,  and  assessments  of  existing  software  products.  Key  to  their 
metrics  approach  is  a  customer  focus.  The  customer  is  defined  as  a  functional  entity  who  looks  at 
the  software  with  a  quality  view  and  has  the  technical  abilities.  Four  rated  levels  are  stated  for 
customer  satisfaction  (ISO  1991).  The  rating  levels  are: 

•  Excellent  -  the  product  equals  or  exceeds  the  requirements  on  every  aspect,  and  can  be  used 
as  specified  -  -  the  product  is  veiy  satisfactory 
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•  Good  -  the  product  equals  or  exceeds  the  requirements  on  most  aspects.  It  can  be  used 

as  specified,  or  with  minor  limitations  —  the  product  is  satisfactory. 

•  Fair  -  the  product  doesn't  meet  the  requirements  on  some  aspects.  It  can,  however,  be 

used,  but  with  major  limitations  -  -  the  product  is  almost  satisfactory. 

•  Poor  -  the  product  does  not  meet  the  requirements.  It  cannot  be  used  -  -  the  product  is 

not  satisfactory. 

We  enhanced  the  features  of  the  ISO  matrix  with  the  addition  of  contingency  questions  if  the 
product  was  rated  Fair  or  Poor.  Babbie  (1979)  defines  contingency  questions  as  questions  in  a 
series  and  whether  they  are  to  be  answered  is  contingent  on  responses  to  the  first  question. 
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Table  5  .  AF-MANCAP  TESTING  SUMMARY 


IfiSl - Ejected  Results _ Test  Resultsl _ Minutes 


1 .  Time  for  10  day  MAMA 

Unknown 

Initial  test 

2.3 

simulation 

Final  test 

2.3 

2.  Time  for  30 day  MAMA 

Unknown 

Initial  test 

5.6 

simulation! 

Final  test 

5.6 

3 .  Time  for  10  day  MANCAP 

Unknown 

Initial  test 

9.45 

simulation 

Final  test 

4.63 

4 .  Time  for  30  day  MANCAP 

Unknown 

Initial  test 

76.43 

simulation 

Final  test 

13.1 

5 .  Time  for  60  day  MANCAP 

Unknown 

Initial  test 

282.0 

simulation 

Final  test 

25.23 

6.  Time  for  180  day  MANCAP 

Unknown 

Initial  test 

Abort 

simulation 

Final  test 

72.53 

7 .  Effect  when  one  AFSC 

Lower  system  availability 

Confirmed 

n/a 

is  constrained^ 

8  Effect  when  three  AFSCs 

Lower  system  availability 

Confirmed 

n/a 

are  constrained^ 

9 .  Effect  when  all  AFSCs 

Lower  system  availability 

Confirmed 

n/a 

are  constrained^ 

10.  Effect  when  spare  parts 

Availability  will  be  lower 

Confirmed 

n/a 

are  constrained3 

1 1 .  Effect  when  on-equipment 
tasks  arc  transferrin^ 

All  workload  transfers 
to  the  next  level 

Confirmed 

n/a 

12.  Effect  of  random  number 
seed  changes 

Results  should  vary  by 
one  standard  deviation 

Confirmed 

n/a 

1 3.  Repeatability  of  model 
with  same  assumptions 

Same  results  should  be 
provided 

Confirmed 

n/a 

1 .  Test  time  shown  in  minutes.  Testing  confirmed  that  simulations  greater  than  30 
days  did  not  change  the  overall  results  of  the  model. 

2.  In  initial  simulations,  the  AFSCs  were  unconstrained  •  value  of  999.  In  a  later 
test,  three  AFSCs  were  reduced  to  the  maximum  demand  level.  For  example, 
with  the  constraint  set  at  21  there  should  be  no  change  in  the  model  availability 
results.  Then  the  manpower  constraint  was  lowered.  For  example,  6  for  the 
peak  demands  or  2  for  the  lowest  possible  demands.  At  these  levels  there 
should  be  a  change  in  the  model  availability  results. 

3.  For  initial  simulations,  the  spares  were  unconstrained  -  value  of  100  percent. 
For  this  test,  the  spares  availability  was  lowered  to  85  percent. 

4.  All  on-equipment  tasks  were  changed  to  off-equipment  tasks. 
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Using  the  above  rating  levels,  the  overall  AF-MANCAP  product  assessments  were  developed 
for  customer  satisfaction  in  five  areas:  functionality,  reliability,  usability,  efficiency,  and  main¬ 
tainability.  Appendix  C  contains  an  example  of  the  survey.  The  customers  ratings  are  covered 
under  results  section  of  this  report. 


Phase  III  -  Convergent  Validation 

The  final  task  in  this  effort  is  the  validation  of  AF-MANCAP.  We  used  a  convergent 
validation  approach.  We  compared  the  results  from  a  proven  model  with  the  results  of  a  new 
model.  This  is  a  predictive  method  to  see  if  the  new  model  outputs  match  the  output  of  an  existing 
model.  The  Logistics  Composite  Model  was  the  proven  model  (Boyle,  1990).  The  AF-MANCAP 
analysis  aid  was  the  new  model.  The  same  database  was  used  for  each  model.  That  database 
focused  on  the  F- 16  on-equipment  listings,  containing  204  maintenance  tasks  associated  with 
eleven  AFSCs.  Next,  a  series  of  common  assumptions  was  developed  for  each  model.  Table  6 
contains  a  list  of  the  common  convergent  validation  assumptions. 


Table  6.  CONVERGENT  VALIDATION  SCENARIO  ASSUMPTIONS 

_ Variable _ Ranee _ 

Number  of  systems  24 

Number  of  simulation  days  30 

Spare  parts  availability  unconstrained 

^SC  availability  unconstrained 

Average  sortie  duration  2  hours 

Turnaround  time  between  sorties  6  hours 

Probability  of  an  abort  occurrence  no  abort 
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Cascio  (1982)  discusses  that  convergent  validation  studies  attempt  to  answer  two  questions: 
(1)  what  is  the  construct  being  measured  by  the  simulation,  and  (2)  how  well  does  the  experiment 
measure  this  construct?  Table  7  lists  the  construct  measurements,  or  what  is  being  measured, 
under  this  experimental  design  for  convergent  validation. 
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Table  7.  EXPERIMENTAL  DESIGN 


Construct 

Ref  Nr.  AF-MANCAP  Measure  (Source) _ LCOM  Measure  (Source) 

1  Maintenance  Headcount  Max/Constrained  Manpower^ 

(MANCAP  Headcount  (Matrix  Post  Processor  Reports) 

Histograms) 


2 .  Average  number  needed  & 

Maximum  number  needed 
(MANCAP  Maintenance  AFSC 
Requirements  Report) 

3 .  Mean  Daily  Maintenance  Hours 
by  AFSC 

(MANCAP  Maintenanc .  Manhour 
Requirements  -  Daily  Ranges  Report) 

4 .  Percent  Utilization  by  AFSC 
(Maintenance  Workload  Report) 

5 .  Percent  System  Availability 
(System  Availability  Repm) 


Average,  Max  Manpower2 
(Matrix  Post  Processor  Reports) 


Daily  Manhour  Requirements2 
(Matrix  Post  Processor  Reports) 


Average  AFSC  Hours 
(Performance  Summary  Report) 

Mission  data 

(Mission  Post  Processor  Reports) 


1.  Maximum  manpower  and  average  manpower  were  taken  from  the  LCOM 
manpower  matrix  reports  and  associated  data.  These  rans  were  unconstrained  on 
manpower  positions  but  consuained  to  the  target  sortie  rates. 

2.  Total  maintenance  hours  are  available  in  the  LCOM  performance  summary  report 
and  mean  daily  maintenance  hours  was  computed  by  dividing  the  total  by  the 
simulated  days. 


SECTION  in  -  RESULTS 
Demonstrations 


Sixteen  officers,  enlisted  and  civilian  personnel  participated  in  the  demonstrations  and 
completed  surveys.  The  military/civilian  breakout  was:  44  percent  civilians,  12  percent  field  grade 
officers,  32  percent  company  grade  officers,  and  12  percent  enlisted.  The  participants'  back¬ 
ground  included:  25  percent  from  Headquarters,  Air  Force  Logistics  Command,  56  percent  fiom 
the  Air  Force  Acquisition  Logistics  Division,  and  19  percent  from  Headquarters,  Tactical  Air 
Command.  Additionally,  the  participants  were  familiar  with  disk  operating  systems,  and  their 
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backgrounds  typically  included  experience  with  human  resource  activities  (e.g.,  preparation  of 
Manpower  Estimate  Reports,  LCOM,  System  Training  Plans,  and  baseline  comparison  system 
analytical  efforts). 

Our  metrics  approach  was  a  customer  focus.  The  customer  is  defined  as  a  functional  entity 
who  looks  at  the  software  with  a  quality  view  and  has  the  technical  abilities.  Four  rating  levels  are 
were  used:  excellent,  good,  fair,  or  poor  (ISO  1991).  The  definitions  linked  to  each  rating  level 
are: 

•  excellent  the  product  equals  or  exceeds  the  requirements  on  every  aspect,  and  can  be 

used  as  specified  -  -  the  product  is  very  satisfactory. 

•  good  the  product  equals  or  exceeds  the  requirements  on  most  aspects.  It  can  be 

used  as  specified,  or  with  minor  limitations  -  -  the  product  is  satisfactory. 

•  fair  the  product  doesn't  meet  the  requirements  on  some  aspects.  It  can, 

however,  be  used,  but  with  major  limitations  -  -  the  product  is  almost 
satisfactory. 

•  poor  the  product  does  not  meet  the  requirements.  It  cannot  be  used  -  -  the 

product  is  not  satisfactory. 

Using  the  above  overall  rating  levels,  16  experts  rated  the  product  in  five  areas:  functionality, 
reliability,  usability,  efficiency,  and  maintainability.  Overall,  the  product  received  a  good  rating. 
Table  8  is  a  summary  of  the  rating  levels.  Also,  the  table  provides  a  breakout  of  responses  by 
three  categories  of  users  (e.g.,  all  users,  non-LCOM  users  and  LCOM  users).  The  non-LCOM 
users  provided  the  highest  overall  product  ratings. 
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Table  8.  SURVEY  RESULTS:  CUSTOMER  SATISFACTION 


Excellent 

Good 

Fair 

Poor 

Functionality 

AU  Users 

0% 

6096. 

20% 

20% 

Non-LCOM  Users 

096. 

8296. 

18% 

0% 

LOOM  Users 

096. 

096. 

25% 

75% 

Reliability 

AU  Users 

8% 

7796. 

15% 

0% 

Non-LCOM  Users 

996. 

91% 

0% 

0% 

LOOM  Users 

096, 

096. 

100% 

0% 

Uscabllity 

AU  Users 

796. 

5796, 

36% 

0% 

Non-LCOM  Users 

996. 

64% 

27% 

0% 

LOOM  Users 

096. 

33% 

67% 

0% 

EfTicicncy 

AU  Users 

096. 

6996, 

15% 

15% 

Non-LCOM  Users 

096. 

82% 

18% 

0% 

LCOM  Users 

0% 

096, 

0% 

100% 

Maintainability 

AU  Users 

2796. 

6496. 

9% 

0% 

Non-LCOM  Users 

33% 

6796, 

0% 

0% 

LCOM  Users 

096. 

5096. 

50% 

0% 

Our  survey  tool  included  questions  on  requirements  for  a  production  version  of  the  product. 
We  provided  a  statement  and  then  asked  each  participant  if  tlicy.  strongly  agree,  agree,  undecided, 
disagree,  or  strongly  disagree  with  the  statement.  We  found  strong  agreement  that  the  Air  Force 
needs  a  baseline  comparison  development  system,  that  depot  level  tasks  should  be  added  to  the 
library,  and  that  the  product  should  be  modified  for  operations  under  WINDOWS  which  would 
provide  more  tasks  for  simulation. 


Our  survey  shows  indecision  concerning  other  product  enhancements.  Some  suppon  was 
shown  for  the  following  features:  fuel  use  linked  to  the  terrain  scenario,  fuel  transportation,  and 
ammunition  transportation.  Overall  agreement  was  lacking  for  the  following  areas:  implementa¬ 
tion  of  K-Kill  computations^,  and  use  of  operational  crew  modeling  feature  of  the  prototype. 
Refer  to  Appendix  D  for  additional  details  of  the  survey  and  the  assessments  for  the  next  update  of 
the  AF-MANCAP  analysis  aid. 


s  To  implement  thi :  feature  requires  the  addition  of  new  library  data  for  the  notional  new 
fighter.  The  model  j^ovides  the  capability  to  identify  the  percent  probability  that  each  component 
will  be  damaged  during  combat,  llie  probabilities  would  be  modeled  and  subsequently  generate 
repair  actions  for  Aircraft  Battle  Damage  Repair  teams. 
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Convergent  Validation 


The  objective  of  this  task  was  first,  to  determine  if  AF-MANCAP  accurately  predicts  the 
outcomes  of  manpower  simulations  and  second,  to  increase  the  users'  confidence  in  the  AF- 
MANCAP  model.  Due  to  the  300  task  limitations,  whole  manpower  requirements  could  not  be 
accomplished.  The  following  results  reflect  manhour  comparisons  of  the  maximum  or  minimum 
task  demands.  Key  to  the  experimental  design  is  the  accepted  use  of  LCOM  as  a  measure  of  the 
real-world  outcome.  Since  LCOM  is  the  Air  Force  standard  (AFM  26-1)  the  results  of  AF- 
MANCAP  can  be  compared  and  any  discrepancies  that  occur  should  be  explainable.  Table  9 
illustrates  the  results  of  the  experimental  design.  The  left  column  shows  the  convergent  validation 
measures,  the  second  column  shows  the  AF-MANCAP  measurement,  and  the  third  column  the 
LCOM  measurement.  The  right  column  shows  the  plus  or  minus  difference  between  AF- 
MANCAP  and  LCO  M. 

Table  9.  CONVERGENT  VALmATION  MEASUREMENTS 


- 

Nr.  Measure 

- EES5K! - 

Measure 

Difference 

Measure 

1 .  Maintenance  Headcount 

65 

54 

-11 

2.  Average  Nr.  Needed/ 

i8/65 

19/54 

+1/-11 

Maxitmun  Nr.  Needed/ 

452X1 

2/8 

2/8 

0/0 

454X3 

6/20 

6/16 

+0/-4 

452X5 

4/18 

5/15 

+1/-3 

458X2 

2/5 

2/5 

0/0 

452X4H 

4/14 

4/10 

(V-4 

3.  Mean  Daily  Maint  His. 

139.78 

145.05 

+5.27 

by  AFSC 

452X1 

15.14 

16.22 

+  1.08 

454X3 

54.39 

55.51 

+1.12 

452X5 

35.62 

36.88 

+1.26 

458X2 

6.34 

7.22 

+0.88 

452X4H 

28.29 

29.22 

+0.97 

4.  Percent  Utilization 

n/a 

n/a 

n/a 

by  AFSC 

452X1 

0.10 

0.067 

-.033 

454X3 

0.26 

0.24 

-.02 

452X5 

0.18 

0.17 

-.01 

458X2 

0.04 

0.03 

-.01 

452X4H 

0.13 

0.13 

0 

5.  Pereent  Operational 

88.81 

90.58 

+  1.77 

Availalnlity 
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SECTION  IV  -  CONCLUSIONS 


Demonstrations 

Sixteen  experts  looked  at  the  AF-MANCAP  product  with  a  view  on  customer  quality.  These 
individuals  possessed  technical  abilities  and  the  decision  power  to  influence  the  potential  use  of  the 
product  The  overall  ratings  were: 

•  functionality  Good  -  the  product  performs  user  required  functions,  with  only  minor 

limitation  in  some  of  them. 

•  reliability  Good  -  the  product  performs  its  intended  functions  under  all  conditions. 

However,  infrequent  conditions  such  as  extreme  load  out  of  specification 
input  data  sets  or  function  parameter  might  result  in  incorrect  output. 

•  useability  Good  -  Minimal  prior  training  is  necessary.  Training  can  be  done  with  a 

"tutorial"  type  of  software  package,  included  in  the  main  delivery.  On-line 
itemized  "help"  is  available.  Full  user  documentation  is  provided. 

•  efficiency  Good  -  response  time  remains  acceptable  under  all  conditions,  however, 

performance  degradation  under  simulation  conditions  is  noticeable.  The 
software  indicated  when  products  will  be  available. 

•  maintainability  Good-  some  documentation  is  available  to  the  user. 

Overall  the  participants  were  undecided  about  the  use  of  the  prototype  now.  Also,  they  were 
undecided  about  the  retention  of  other  features  in  the  model  (e.g.,  operational  crew  modeling,  fuel 
consumption,  and  the  logistics  report  features).  Several  wanted  more  "hands  on  time"  and 
additional  technical  documentation  about  the  simulation  networic. 

Convergent  Validation 

Once  the  model  was  tested  and  the  convergent  validation  results  obtained,  the  meaning  must 
be  interpreted  back  into  the  real  world.  Based  on  these  results,  AF-MANCAP  nearly  mirrored  the 
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manhour  requirements  predictions  from  LCOM,  but,  due  to  software  constraints,  AF-MANCAP 
could  not  predict  LCOM’s  whole  manpower  requirements. 

AF-MANCAP’s  prediction  success  for  Mean  Daily  Maintenance  Hours  (Table  9,  Ref.  Nr.  3) 
shows  that  the  underlying  simulation  generates  maintenance  and  accounts  for  task  manhour 
requirements  accurately.  The  Average  Number  Needed  (Table  9,  Ref  Nr.  2)  and  Percent 
Utilization  by  AFSC  (Table  9,  Ref.  Nr.  4)  are  likewise  derived  from  the  manhour  requirements 
and  are  also  favorable.  The  Percent  Operational  Availability  data  (Table  9,  Ref.  Nr.  5)  also 
compared  favorably  since  it  is  task  length  dependent.  However,  the  favorable  comparison  is  not 
true  for  the  Maintenance  Headcount  (Table  9,  Ref.  Nr.  1). 

The  Maintenance  Headcount  or  manpower  prediction  capabilities  of  AF-MANCAP  require 
evaluation  of  peak  whole  manpower  requirements  supporting  a  targeted  mission  rate.  Each  AFSC 
would  have  to  be  constrained  (numbers  of  positions  available  raised  or  lowered)  until  each  AFSC 
proportionally  contributed  to  the  targeted  mission  rate.  This  could  not  be  accomplished  using  the 
present  AF-MANCAP  software  due  to  the  limitation  of  300  tasks  per  BCS  and  the  inherent 
differences  between  AF-MANCAP  and  LCOM  mission  processing  (See  SECTION  V).  Some 
peak  manpower  differences  showed  in  the  Maintenance  Headcount  (Table  9,  Ref.  Nr.  1).  Had 
constraining  been  possible,  differences  would  have  also  shown  in  the  Maximum  Number  Needed 
(Table  9,  Ref.  Nr.  2).  As  illustrated,  the  AFSC  specific  elements  from  (Table  9,  Ref.  Nr.  2)  are 
all  that  is  shown.  Even  that  data  shows  different  peak  demand  relationship. 

Applicability  of  AF-MANCAP  to  the  Acquisition  Process 

Tetmeyer  and  Moody  (1974)  indicate  that  the  following  characteristics  apply  to  manpower 
tools: 

V  Should  provide  early  estimates  for  use  in  trade-offs  and  evaluation 

V  Should  be  sensitive  to  the  way  in  which  the  new  system  wiU  be  employed 

The  AF-MANCAP  analysis  aid  supports  some  early  phases  of  an  acquisition  when  the  using 
command  is  striving  to  identify  manpower  drivers  and  manpower  constraints.  The  analysis  aid  can 
provide  inputs  for  the  preparation  of  the  Mission  Need  Statement  and  Operational  Requirements 
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Document.  Also,  the  AF-MANCAP  analytical  approach  suppons  manhour  estimation  activities 
conducted  by  an  acquisition  activity  during  the  concept  development  phase. 

The  AF-MANCAP  analysis  aid  is  not  as  sensitive  as  LCOM  to  the  way  a  system  will  be 
employed.  The  AF-MANCAP  simulation  scenario  does  not  permit  the  development  of  staggered 
launch  or  varying  sortie  lengths.  All  systems  fly  and  all  systems  enter  maintenance  at  specified 
times.  The  lack  of  a  staggered  or  variable  sortie  length  currently  reduces  the  utility  of  the  product 
for  detailed  on-equipment  comparability  analytical  efforts. 

With  enhancements,  the  AF-MANCAP  analysis  aid  could  be  used  for  baseline  comparison 
system  development  during  the  early  acquisition  phase.  The  analysis  tool  provides  the  flexibility  to 
transition  from  the  baseline  comparison  system  to  a  proposed  system  construct  development.  In 
this  form  of  use  it  supports  early  trade-off  studies  where  manhours  or  systems  availability  factors 
are  the  measures  of  merit. 


SECTION  V  -  RECOMMENDATIONS 

In  our  demonstrations  and  discussions,  we  found  potential  users  for  the  tool  in  the  present 
configuration.  First,  the  tool  could  be  used  to  support  two  levels  of  maintenance  studies  on 
avionics  systems  for  Air  Force  Logistics  Command.  Second,  Manpower,  Personnel,  and  Training 
program  managers  could  use  the  analysis  aid  for  trade-off  evaluations  associated  with  common 
avionics  developments.  We  recommend  that  the  current  model  be  provided  to  rhe  ahnv#*  artivities 
for  utilization  with  small  (300  task)  databases.  The  model  can  provide  top  level  trade-off  support 

Throughout  our  demonstrations,  we  noted  other  features  that  should  be  added.  For  example, 
personnel  at  Headquarters,  Tactical  Air  Command  expressed  an  interest  in  the  model  if  it  included 
life  cycle  cost  trade-off  computations.  Another  critical  improvement  is  needed  in  the  number  of 
components  model.  The  task  capability  of  the  model  should  be  expanded  from  300  to  3,000  tasks, 
and  the  model  should  be  modified  to  allow  sortie  scheduling  sequences  plus  mission  specific  sortie 
lengths.  During  the  demonstrations  and  testing  we  noted  other  product  limitations.  These  are 
discussed  in  Appendix  A:  AF-MANCAP  Explained. 

This  study  assessed  the  feasibility  of  adapting  part  of  the  Army's  HARDMAN  HI  technol¬ 
ogy  for  Air  Force  use.  Our  software  modifications  were  successful.  We  achieved  the  objective  of 
completing  a  convergent  validation  of  the  AF-MANCAP  model  with  LCOM.  This  evaluation 


31 


produced  a  working  prototype.  The  working  prototype  and  extensive  documentation  reduce  the 
risk  of  future  product  developments.  We  believe  the  prototype  is  robust  enough  to  serve  as  a  low- 
risk  baseline  for  future  product  evolution.  Table  10  provides  a  list  of  our  recommended  features 
for  a  production  version  of  the  AF-MANCAP  analysis  aid.  We  believe  these  are  low-risk  changes 
and  we  recommend  a  succeeding  cycle  of  product  development. 

Table  10.  LISTING  OF  FEATURES  FOR  THE 
PRODUCTION  ANALYSIS  AID 


Ref  Nr  Feature _ 

1.  Convert  from  DOS  to  WINDOWS  which  will  allow  for  the  expansion  of  task 
capabilities  from  300  to  3,000 

2 .  Expand  the  library  to  include  depot  level  tasks 

3.  Conduct  further  review  of  the  need  to  retain  such  features  as:  fuel  consumption  and 
transportation  vehicle  use,  ammunition  consumption  and  transportation,  probability  of 
kill,  and  probability  of  abort. 

4.  Allow  the  user  to  define  pre-  and  post-sortie  maintenance  tasks. 

5.  Allow  the  user  to  specify  the  sortie  scheduling  sequences  and  mission  specific  sortie 
lengths. 

6 .  Provide  variance  capability  for  the  mean  time  to  repair  data. 

7.  Provide  a  report  of  the  sorties  flown  and  the  flying  hours. 

8.  Add  a  new  utilities  function  that  would  provide  the  capability  to  read  ASCIfi  formatted 
files  and  query  the  user  regarding  where  the  data  would  be  placed  in  the  library  file 
structure. 

9.  Add  new  utilities  functions  to  provide  for  the  export  of  data  to  the  Logistics  Support 
Cost  Model. 

1 0.  Allow  the  user  to  enter  the  percentage  of  indirea  time  per  shift. 

1 1 .  Provide  for  pre-screening  of  histograms  to  preclude  the  printing  of  all  histograms  when 
no  data  were  generated  during  the  simulation. 

12.  Eliminate  the  SPARC?  interfaces  by  allowing  the  user  to  enter  R&M  20003  like  goals. 

1 3 .  Link  the  product  to  a  life  cycle  cost  (examples  are  SABLE^  or  CORE5). 

14.  Provide  recommended  changes  to  adjust  the  mean  operational  units  between  failure 
(MOUBF)  when  the  operational  environment  changes. 


1  American  Standards  for  Communications  Information  Interchange 

2  System  Performance  And  Reliability  Criteria 

3  Reliability  and  Maintainability  2000 

4  Systematic  Approach  to  Better  Long-range  Estimating 

5  Ctost  Oriented  Resource 

In  summary,  the  customer-focused  product  survey  proved  to  be  a  good  customer  feedback 
device.  This  survey  was  combined  with  a  demonstration  of  the  product  with  the  use  of  a  liquid 
crystal  display  placed  on  top  of  an  overhead  projection.  Through  these  techniques  we  increase  our 
ability  to  illustrate  the  feature  of  the  product  during  a  fast-paced  two  hour  demonstration.  In 
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addition  the  AF-MANCAP  Demonstration  Handbook  (Flint  and  Dahl,  August  1991)  provides  the 
user  key  "how  to"  examples. 

In  retrospect,  we  could  have  improved  the  demonstration  phase  of  the  project  with  the  use  of 
a  background  paper  describing  the  model.  Providing  the  paper  to  the  participants  before  the 
demonstration  could  have  reduced  the  number  of  questions.  After  the  demonstration,  we  created  a 
technical  paper  that  responds  to  the  typical  questions  (Flint  and  Dahl,  December  1991). 

The  prototype  of  the  AF-MANCAP  analysis  aid  has  undergone  an  extensive  review.  To  our 
knowledge  this  was  the  first  time  that  the  Armstrong  Laboratory  (AL/HRMM)  has  applied  a 
convergent  validation  strategy  to  a  manpower  model.  The  results  were  encouraging.  With 
modification,  the  AF-MANCAP  analysis  aid  will  be  a  powerful  PC-based  simulation  tool  for  the 
acquisition  community.  The  enhanced  tool  would  support  the  development  of  Baseline  Com¬ 
parison  Systems  and  the  subsequent  trade-off  studies.  As  an  example,  the  trade-off  capability 
would  support  the  evaluation  of  changes  in  system  parameters  such  as  mean  time  to  repair  or  mean 
time  between  failure.  Each  parameter  could  be  reviewed  independently  in  terms  of  effects  on  each 
other.  AF-MANCAP  provides  the  added  feature  of  integrating  the  elements  with  a  user-defined 
scenario.  The  mathematical  and  simulation  features  of  the  model  make  it  possible  to  deal  with  the 
variables  of  the  problem  on  a  simultaneous  bases  and  provide  relative  measures  of  merit  indicators 
in  terms  of  higher-order  system  parameters  such  as  system  availability. 
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SECTION  VII  -  GLOSSARY 


Achieved  Availability 


Inherent  Availability 


IMPACTS 


LCOM 


Achieved  availability,  within  AF-MANCAP,  is  calculated  as  the 
percentage  of  time  the  system  was  able  to  go  out  on  its  scheduled 
missions.  If  the  system  missed  a  mission  because  the  maintenance 
actions  were  not  complete,  then  the  Achieved  Availability  percen¬ 
tage  will  be  reduced.  If  the  system  has  an  Achieved  Availability  of 
100%,  then  it  never  missed  a  mission. 

Inherent  availability,  within  AF-MANCAP,  is  calculated  as  shown 
in  the  following  algorithm: 

Ai=  MTBF/(MTBF-^MTTR) 

where: 


MTBF  =  Mean  time  between  failure  for  the  system 

MTTR  =  mean  time  to  repair  the  system 

Integrated  Manpower,  Personnel  and  Comprehensive 
Training  &  Safety  -  An  acquisition  management  program  that 
directly  implements  the  specific  human  systems  integration, 
manpower,  personnel,  training,  health  hazards,  safety,  and  human 
factors  engineering  policy  outlined  in  DoDD  5000.1,  DoDI  5000.2, 
AFR  57-1,  and  AFR  57-4  into  the  Air  Force  Acquisition  process 
and  functional  managers. 

Logistics  Compc^ite  Model  -  a  highly  flexible  and  versatile 
model  used  to  simulate  the  work  of  a  maintenance  organization.  It 
may  be  applied  to  suppon  widely  varying  study  objectives,  but  is 
most  often  applied  to  identify  the  optimal  mix  of  logistics  resources 
to  support  a  given  weapon  system  under  given  operating  conditions. 
Logistics  resources  considered  can  include:  spare  pans,  suppon 
equipment,  facilities,  or  human  resources. 
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Operational  Availability  Operational  availability,  within  AF-MANCAP,  is  defined  as 

follows: 

Ao  =  (OT  +  ST)  /  (OT  +  ST  +  TCM  +  TPM  +TALDT) 
where: 

OT  =  Operating  time  during  a  given  calendar  time  period 
ST  =  Standby  time  (Not  operating  but  assumed  operable) 

TCM  =  Total  corrective  maintenance  downtime  in  clock  hours 
during  the  given  time  period 

TPM  =  Total  preventative  maintenance  downtime  in  clock  hours 
during  the  stated  OT  period 

TALDT=  Total  administrative  and  logistics  downtime  spent 
waiting  for  parts,  maintenance  personnel,  or  transporta¬ 
tion  per  given  calendar  time  period 
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APPENDIX  A:  AF-MANCAP  EXPLAINEDi 


Introduction 

The  purpose  of  the  Air  Force  MAHpower  CAPabilities  Tool  (AF-MANCAP)  is  to  allow 
Air  Force  analysts  to  evaluate  maintenance  and  supply/support  personnel  requirements  for 
proposed  new  weapon  systems  at  a  variety  of  unit  levels.  The  AF-MANCAP  modeling  tool 
enables  analysts  to  model  units  and  determine  the  requirements  for  manpower  based  on  equipment 
reliability  and  maintainability  (R  &  M)  characteristics,  maintenance  concept,  and  supply  concept. 

AF-MANCAP  is  the  Air  Force's  version  of  the  MANCAP  n  tool  that  is  part  of  the  Army 
Research  Institute's  HARD  ware  vs.  MANpower  (HARDMAN  HI)  project.  The  MANCAP 
approach  was  originally  inspired  by  the  Air  Force's  Logistics  Composite  Model  (LCOM), 
currently  managed  by  the  Air  Force  Management  Engineering  Agency  (AFMEA)  and  used 
throughout  the  Air  Force.  AF-MANCAP  is  intended  to  analyze  a  subset  of  the  issues  encompassed 
by  LCOM,  and  to  provide  a  PC-based  tool  for  logistics  and  manpower  analysts  to  perform  an 
evaluation  in  a  timely  manner.  It  is  not  intended  to  replace  LCOM,  rather  it  provides  a  unique 
feature  that  enables  MPT  analysts  to  develop  a  Baseline  Comparison  System  (BCS)  for  beginning 
manpower  analysis  prior  to  development  of  full  LCOM  capability. 

AF-MANCAP  helps  analysts  determine  manpower  requirements  for  a  system  at  a  variety 
of  unit  levels.  AF-MANCAP  takes  into  consideration  maintenance  requirements  for  scheduled 
maintenance,  unscheduled  maintenance,  and  battle  damage  repair  at  different  levels  of  combat 
activity.  These  requirements  are  represented  in  AF-MANCAP  in  terms  of: 

1)  the  Operational,  Achieved,  Inherent,  and  Sortie  availabilitv  of  the  systems  in  the 
unit  given  the  configuration  of  unit  personnel 

2)  the  total  number  of  maintenance  manhours  and  headcount  required  for  each  AFSC 
at  each  maintenance  level 


1  Authors:  Dahl,  S  &  Flint,  R.  (December  1991).  This  paper  introduces  a  general  audience 
to  AF-MANCAP. 
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3)  the  utilization  or  workload  of  maintenance  personnel  at  each  level 


The  methodology  that  AF-MANCAP  uses  to  estimate  personnel  requirements  is 
stochastic  network  simulation  modeling.  At  the  heart  of  the  AF-MANCAP  tool  is  a  generic 
maintenance  simulation  model  that  can  represent  any  new  weapon  system.  Analysts  can  supply 
and  modify  model  parameters  as  input  for  the  simulation  model.  Data  can  be  obtained  from  AF- 
MANCAP  libraries  of  various  system  types  or  from  other  U.S.  Air  Force  sources. 

The  current  version  of  AF-MANCAP  has  retained  much  of  the  original  functionality 
developed  for  the  Army  Research  Institute's  version.  As  an  example,  maintenance  data  for  21 
Army  systems  are  included  in  the  AF-MANCAP  liteary. 

The  Three  Parts  of  the  Analytical  Tool 

This  tool  c  insists  of  three  parts.  The  first  part  is  used  to  develop  a  Baseline  Comparison 
System  (BCS).  The  second  part  is  used  to  model  a  single  system's  maintenance  requirements. 
The  third  part  is  used  ro  model  multiple  systems'  maintenance  requirements.  Each  of  these  parts  is 
briefly  discussed  in  this  section. 

Part  1  :  The  Baseline  Comparison  System  Developer  -  -  In  this  part,  users 
develop  a  description  of  the  tasks  required  to  maintain  each  component  in  their  new  system,  how 
frequently  each  task  needs  to  be  performed,  and  what  kinds  of  maintainers  are  needed  to  perform 
those  tasks.  The  tool  also  helps  users  define  mission  scenarios  that  include  the  amount  that  each 
subsystem  is  used  anc  the  amount  of  time  that  is  available  for  maintenance  given  system 
availability  requirements 

This  part  is  designed  to  enable  users  to  "cut  and  paste"  all  of  this  information  from  a 
database  of  existing  system  data  into  the  description  of  a  new  system.  Users  can  mix  data  from 
several  different  existing  systems,  at  either  the  subsystem  or  component  level.  In  addition,  users 
can  enter  completely  new  data  into  the  description. 

The  output  of  the  I:  CS  Developer  provides  the  other  two  pans  of  the  tool  with  com¬ 
ponent  maintenance  parameten  These  component  maintenance  parameters  include: 
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•  MOUBF  (Mean  Operational  Units  Between  Failure)  for  each  maintenance  action2 

•  MTTR  (Mean  Time  to  Repair) 

•  Who  does  the  maintenance  action  (up  to  two  different  Air  ^orce  Specialty  Codes) 

•  The  organizational  level  at  which  the  maintenance  action  is  performed 

•  The  probability  that  the  need  for  the  maintenance  action  will  cause  a  mission  abort 


Part  2  :  The  Single  System  Model  -  AF-MAMA  -  -  The  second  part  of  the  tool  is 
a  version  of  the  Army's  MAintenance  MAnpower  Analysis  Aid  (MAMA).  This  waS  renamed  Air 
Force  MAintenance  MAnpower  Analysis  Aid  (AF-MAMA)  and  is  a  stochastic  model  that  is  used  to 
predict  the  maintenance  requirements  cf  a  single  system.  AF-MAMA  is  a  pre-processor  and  must 
be  performed  prior  to  use  of  Part  3  of  the  analysis  aid. 

For  all  maintenance  actions  that  do  not  lead  to  mission  aborts,  the  occurrences  are 
generated  from  the  MOUBF  parameter  entered  in  the  BCS  Developer  and  passed  through  to  AF- 
MAMA.  For  maintenance  actions  that  do  lead  to  mission  abort  the  occurrences  are  generated  using 
data  pre-processed  by  AF-MAMA.  For  those  data,  AF-MAMA  generates  the  mean  time  between 
failure  for  each  subsystem  and  the  probability  of  each  possible  maintenance  action  given  a 
subsystem  failure.  These  calculated  failure  rates  are  then  passed  on  to  the  multiple  systems  model. 

A  detailed  discussion  of  AF-MAMA  is  available  in  Dahl,  Adkins,  and  Bravo  (1990). 
The  remainder  of  this  paper  only  addresses  the  multiple  systems  modeling  tool. 

Part  3  :  The  Multiple  Systems  Model  -  AF-MANCAP  -  -  This  final  part  of  the 
tool  includes  a  task  network  model  that  represents  multiple  systems  performing  missions  and 
undergoing  maintenance.  This  part  is  discussed  in  detail  in  the  following  pages. 

AF-MANCAP  Simulation  Overview 

AF-MANCAP  Inputs  -  -  The  AF-MANCAP  model  is  driven  by  data  entered  using  a 
series  of  pop-up  screens.  The  user  can  supply  and  modify  model  parameters  as  input  for  the 
simulation  model.  These  parameters  include: 


2  A  maintenance  action  is  a  maintenance  task/component  pair  (e.g.,  remove  &  replace 
engine). 


A-4 


43 


•  System  usage  rates  -  These  are  specified  in  rounds  fired  by  each  gun  per  hour, 
distance  traveled  per  hour,  and  amount  the  system  was  operated  each  hour.  These 
values  are  used  to  accrue  usage  on  the  components  in  the  system.  This  ultimately 
drives  the  frequency  of  failure  for  each  component 

•  Number  of  available  maintainers  -  This  is  specified  by  AFSC  and  by  maintenance 
organization  level.  The  number  of  available  maintainers  will  affect  the  availability 
of  the  systems. 

•  Combat  activity  levels  -  The  user  enters  a  mission  profile  by  specifying  different 
activity  levels,  and  sequencing  those  levels  throughout  the  simulation.  The  model 
accrues  usage  on  the  comptHients  at  different  rates  for  each  activity  level. 

•  Nuiaber  of  systems  to  be  modeled  -  The  maximum  number  of  systems  AF- 
MANCAP  can  model  ir  each  simulation  run  is  500. 

Users  arc  also  asked  to  enter  the  number  of  days  that  they  want  the  simulation  to  run,  the 
travel  time  between  different  levels  of  maintenance,  and  the  probability  that  spare  parts  will  be 
available  when  needed. 
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How  AF-MANCAP  Works 


AF-MANCAP  is  a  sophisticated  tool  that  includes  a  task  network  simulation 
model.  This  model  is  quite  complicated  and  is  fully  documented  in  the  AF-MANCAP  Software 
Documentation  Technical  Report.  In  this  appendix,  we  attempt  to  simplify  the  discussion  of  how 
the  model  works  and  how  the  factors  in  the  model  play  together  to  predict  maintenance  re¬ 
quirements. 

AF-MANCAP  is  essentially  a  limited  resources  model  that  is  designed  to  simulate 
maintenance  organizations.  AF-MANCAP  is  based  on  a  task  network  model.  The  model  sends 
systems  out  to  perform  missions,  and  accrues  usage  on  all  components  at  a  rate  appropriate  for  the 
equipment  type  and  the  activity  level. 

For  maintenance  actions  that  do  not  cause  the  current  mission  to  abort  when  they  fail,  AF- 
MANCAP  uses  the  MOUBFs  and  an  exponential  distribution  to  predict  when  the  failure  will 
occur.  For  maintenance  actions  that  do  cause  the  current  mission  to  abort,  AF-MANCAP  uses 
probabilistic  distributions  compiled  in  the  AF-MAMA  pre-processor  to  predict  failures.  At  the  end 
of  each  mission,  each  system  that  experienced  failures  is  sent  to  maintenance. 

In  maintenance,  the  systems  are  sorted  such  that  those  that  can  be  repaired  in  the  shortest 
time  are  placed  in  maintenance  first.  Then,  the  maintenance  is  modeled  by  applying  the  user- 
entered  limitations  of  maintenance  crew  size  by  AFSC.  When  each  system  exits  from  maintenan¬ 
ce,  it  is  inserted  back  into  the  mission  schedule. 

More  Details  -  -  The  AF-MANCAP  model  takes  preprocessed  information  from  the  AF- 
MAMA  model  and  new  user  input  scenario  and  combat  level  information  as  input  and  simulates 
maintenance  requirements  across  multiple  systems.  Figure  1  illustrates  the  top  level  of  the  AF- 
MANCAP  model.  The  model  is  composed  of  frve  main  parts  that  are  discussed  in  detail  below 
(NOTE:  Rectangles  in  the  diagram  denote  networks  of  tasks,  ovals  denote  tasks.  The  numbers 
inside  each  symbol  are  used  to  identify  the  task  or  network  and  roughly  correlate  to  processing 
order.): 


1)  The  start  task  initializes  all  model  counters  and  synchronously  starts  four  of  the 
five  main  parts. 
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2)  The  increment  periods  task  updates  the  24-hour  day  counter  and  calculates  the 
daily  availability  measurements  at  the  end  of  each  24  hours. 

3)  The  increment  intervals  task  changes  the  combat  level  based  on  the  clock 
duration. 

4)  The  manpower  pool  network  refreshes  the  available  hours  for  each  AFSC  at  the 
shift  change.  At  the  end  of  the  simulation,  the  mean  maintenance  hours  per  AFSC 
per  day  and  the  standard  deviation  are  calculated. 


Figure  1.  AF-MANCAP  top  level  network  diagram. 


5)  The  make  failures  network  feeds  the  systems  into  the  scheduled  missions. 
Systems  then  begin  missions  that  last  until  the  next  failure  time  scheduled  for  that 
individual  system.  When  a  system  finishes  a  mission,  maintenance  tasks  are 
generated  and  routed  to  maintenance.  If  a  system  is  killed,  a  replacement  system  is 
introduced  after  a  user-specified  length  of  time. 

6)  The  perform  maintenance  network  routes  tasks  to  the  Aircraft  Battle  Damage 
Repair  (ABDR)  team,  On-Equipment,  Off-Equipment,  or  Depot.  Travel  times  to 
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maintenance  and  between  maintenance  levels  as  well  as  pans  delay  times  are 
calculated.  Maintenance  takes  place  based  on  which  system  can  be  repaired  most 
quickly  with  off-line  tasks  as  lowest  priority.  Systems  are  returned  to  the  current 
mission  as  soon  as  they  are  repaired. 

Figures  2  through  4  show  the  expanded  views  of  the  three  main  networks.  The 
foUowing  discussion  briefly  explains  the  activities  that  are  performed  in  these  tasks. 


1.1 _  1.2 _ 
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Figure  2.  Manpower  pool  network. 
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Figure  3.  Make  failures  network. 
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Figure  4.  Perform  maintenance  network. 

Task  Start  -  -  This  task  initialize  all  model  counters  and  synchronously  starts  four  of 
the  five  main  parts.  This  task  does  not  require  the  model  clock  to  increment  Initialization  events 
are  "zero  time"  task.  This  task  is  followed  by  two  tasks,  1.3  (increment  periods)  and  1.5 
(increment  intervals)  and  by  two  networks,  110  (Manpower  Pool)  and  300  (Make  Failures). 

Task  1.3  •  Increment  Periods  -  -  This  task  counts  the  number  of  full  days  simulated 
and  completes  the  daily  calculations.  The  task  cycle  time  is  one  full  day.  In  this  task,  AF- 
MANCAP  increments  the  overall  mission  time  by  the  mission  time  accrued  in  this  period  by  all  the 
systems,  calculates  the  daily  operational  availability  and  increments  the  daily  maintenance  hours 
and  the  sum  of  the  squares  of  the  maintenance  hours  for  each  AFSC  at  each  level.  This  task  is 
followed  only  by  itself.  The  period  counter  just  keeps  cycling  every  day  until  the  end  of  the 
simulation  time. 


Task  1.5  -  Increment  Intervals  -  -  An  interval  is  the  amount  of  time  missions  are 
held  at  a  single  combat  activity  level.  In  AF-MANCAP,  there  are  only  two  combat  activity  levels 
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currently  being  used^.  The  two  levels  cuirently  being  used  are  "Maintenance"  and  "Fly."  For 
example,  this  task  increments  the  interval  counter  at  the  end  of  the  Maintenance  interval  and  starts 
the  next  Fly  interval.  If  the  interval  is  changing  from  Maintenance  to  Fly,  AF-MANCAP  calculates 
the  sortie  rate  availability.  This  task  cycles  back  to  itself  for  the  duration  of  the  simulation. 

Network  110  -  Manpower  Pool  -  -  The  manpower  pool  network  refreshes  the 
available  hours  for  each  AFSC  at  the  shift  change.  At  the  end  of  the  simulation,  the  mean 
maintenance  hours  per  AFSC  per  day  and  the  standard  deviation  are  calculated.  Tasks  1.1 
(Change  Shift)  and  1.2  (Collect  Data)  are  contained  in  this  network. 

Task  1.1  -  Change  Shift  -  -  This  task  refreshes  the  manpower  pool  by  reallocating 
hours  to  each  AFSC  at  each  level  at  the  shift  change.  The  length  of  a  shift  is  24  hours  divided  by 
the  number  of  shifts  specified.  This  task  cycles  back  to  itself  until  the  end  of  the  simulation  at 
which  time  1.2  (Collect  Data)  is  called. 

Task  1.2  -  Collect  Data  -  -  This  is  a  zero  time  task.  For  each  AFSC  at  each  level  the 
mean  maintenance  hours  and  standard  deviation  are  calculated  from  the  sum  and  sum  of  the 
squares.  This  is  the  final  task  executed.  None  follow  it 

Network  300  -  Make  Failures  -  -  This  network  initially  generates  all  the  systems. 
Systems  then  begin  to  perform  their  mission.  When  a  system  finishes  a  mission,  maintenance 
tasks  are  generated  and  routed  to  maintenance.  If  a  system  is  killed,  a  replacement  system  is 
introduced  after  a  specified  length  of  time.  There  are  five  tasks  in  this  network:  301  (Generate 
Systems),  302  (Perform  Mission),  304  (Pick  Tasks),  305  (Route  to  Maintenance),  and  306  (Wait 
Kill  Time). 

Task  301  -  Generate  Systems  -  -  System  tags  are  created  and  sent  into  the  first 
mission.  All  the  systems  begin  the  first  mission  at  the  same  time.  This  task  returns  to  itself  until 
all  the  systems  are  generated.  The  schedule  for  operational  requirements  is  obtained  from  the 
Sequence  Combat  Activities  screen  that  is  input  by  the  analyst.  Currently,  the  scheduling  sequence 
is  limited  to  hourly  increments  over  a  24  hour  period.  Generated  systems  fly  at  the  start  of  the 


3  Users  can  enter  and  up  to  five  different  activity  levels.  Users  have  the  opportunity  to  enter 
names  for  each  activity  level  (e.g.,  alert,  recovery). 
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hour  and  recover  at  the  end  of  the  hourly  increment  specified  by  the  analyst.  The  generated 
systems  are  sent  to  302  (Perform  Mission). 

Task  302  -  Perform  Mission  -  -  All  available  systems  begin  the  first  mission  at  the 
same  time.  If  abort  data  or  K-Kill  data  are  not  used,  all  systems  end  the  first  mission  at  that  same 
time.  The  software  also  models  each  system  independently. 

This  task  involves  "looking  ahead"  to  determine  what  the  next  failure  will  be  for  each 
system.  There  are  two  types  of  failures; 

1)  Abort  failures  are  generated  by  using  failure  probabilities  that  were  generated  during 
execution  of  the  single  system  model  (AF-MAMA).  AF-MAMA  drew  failure  rates 
for  each  component  fi'om  an  exponential  distribution  and,  over  the  length  of  the 
simulation,  calculated  the  failure  rate  of  each  subsystem.  AF-MAMA  then 
calculated  the  probability  that  each  component  in  the  subsystem  caused  the  failure. 

2)  Nonabort  failures  are  generated  at  the  component  level  by  drawing  from  each 
individual  component's  failure  distribution  (i.e.,  an  exponential  distribution  with 
the  component's  MOUBF  as  the  mean). 

After  the  next  failure  is  identified,  there  are  three  possibilities; 

1)  If  the  user  specified  (during  the  development  of  the  mission  profile)  that  nonaborts 
are  to  be  repaired  on  occurrence  during  the  Fly  activity  level,  then  the  next  failure  is 
generated.  This  failure  can  be  either  an  abort  or  nonabort.  This  ntission  is 
advanced  to  that  failure  time  and  then  the  system  is  pulled  into  maintenance  when 
failure  time  is  reached. 

2)  If  the  user  specified  (during  the  devel(q)inent  of  the  mission  profile)  that  nonaborts 
are  to  be  repaired  only  when  the  system  is  in  maintenance,  the  mission  is  completed 
and  then  the  system  is  pulled  into  maintenance  at  the  end  of  the  mission  or  other 
operational  activity  levels. 

3)  If  the  user  specified  abon  or  K-kill  data  (during  the  database  development)  and 
aborts  are  turned  "on"  (during  the  development  of  the  simulation  parameters)  then 
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aborts  are  to  be  repaired  on  occurrence  during  the  Fly  activity  or  other  operational 
levels  specified  by  the  analyst.  Also,  combat  damage  repairs  are  generated  on 
occurrence  during  the  Fly  or  other  operational  activity  levels.  The  system  is  pulled 
into  maintenance  when  the  failure  time  is  reach  for  the  abort  or  combat  damage 
repair.4 

Once  the  new  failure  is  generated,  the  system  can  be  routed  to  one  of  the  following 

nodes: 

1)  If  the  survivability  parameters  dictate,  the  system  can  be  killed  and  then  sent  to  306 
(Wait  K-Kill). 

2)  If  the  system  is  not  killed  and  needs  maintenance,  it  is  sent  to  304  (Rck  Tasks). 

3)  If  no  failures  were  generated  during  the  mission  and  the  system  was  not  killed,  the 
system  is  returned  to  302  (Perform  Missions)  and  waits  there  until  the  Fly  or 
operational  segment  is  scheduled. 

All  "in  commission  systems"  wait  until  the  next  operational  period  (defined  by  the  analyst 
through  the  Sequence  Combat  Activities  and  Define  Operational  Mode  Summary /Mission  Profile 
windows).  They  also  complete  missions  as  a  group  (unless  some  of  them  have  experienced 
failures,  combat  damage,  or  K-kill). 

Task  304  -  Pick  Tasks  -  -  Specific  tasks  are  generated  here  and  sent  into  the 
maintenance  network.  There  are  two  different  kinds  of  tasks  that  can  be  generated,  those  that  do 
not  lead  to  mission  abort  ("nonaborts")  and  those  that  lead  to  the  interruption  of  the  current  mission 
("aborts").  In  addition,  tasks  can  either  be  caused  by  component  reliability  parameters  or  by 
combat  damage.  Therefore,  there  are  four  different  types  of  maintenance  tasks.  These  are 
described  below: 


4  A  system  attrition  (input  during  the  development  of  the  system  survivability  parameters)  can 
occur  if  the  probalnlity  of  a  K-kill  is  greater  that  0.0%.  Attrition  is  pulled  from  a  uniform 
distribution.  Once  the  attrition  is  generated  the  system  is  no  longer  available.  Replacements  can  be 
added  (input  during  the  development  of  the  system  replacement  lag  time,  specified  in  hours).  The 
replacements  are  not  available  until  the  lag  time  has  accumulated  after  the  K-kill. 
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1)  Nonabort  failures  caused  bv  reliability  parameters  -  AF-MANCAP  predicts  these 
failures  by  comparing  the  system  usage  since  the  last  time  each  nonabort  component 
failure  was  generated  to  failure  times  picked  from  exponential  distributions  whose 
means  are  the  component  MOUBFs.  These  are  the  same  MOUBFs  that  are  input 
data  for  the  AF-MAMA. 

The  Notional  Tactical  Fighter  aircraft  components  in  the  AF-MANCAP  library  do 
not  include  any  components  failure  data  that  will  generate  mission  aborts.  Also, 
they  do  not  include  any  combat  damage  probabilities  or  K-kill  probabilities.  For 
this  reason,  the  remaining  three  types  of  task  generation  do  not  apply  to  the  current 
Air  Force  databases. 

2)  Nonabort  failures  caused  bv  combat  damage  -  AF-MANCAP  predicts  these  failures 
by  comparing  random  number  picks  against  the  joint  probability  that  a  component 
will  experience  a  combat  hit  and  that  a  component  will  not  cause  an  abort. 

3)  Abort  failures  caused  bv  reliability  parameters  •  These  failures  are  predicted  using 
data  pre-processed  by  the  AF-MAMA  model.  AF-MAMA  generates  probabilities 
that  each  subsystem  will  experience  a  failure  that  causes  a  mission  to  abort. 

4)  Abort  failures  caused  bv  combat  damage  -  ese  failures  arc  predicted  by  com¬ 
paring  random  number  picks  against  the  ^lobability  that  a  component  will 
experience  a  combat  hit  and  the  probability  that  the  component  will  cause  an  abort. 

If  no  tasks  are  generated,  the  system  is  returned  to  302  (Perform  Mission).  Otherwise, 
the  system  waits  until  repairs  are  ccmiplete. 

Task  306  -  Wait  kill  time  -  -  If  a  system  is  killed,  another  system  is  reintroduced 
after  a  user-defined  length  of  time.  The  new  system  is  sent  to  302  (Perform  Mission). 

Network  200  -  Perform  Maintenance  -  The  routing  of  tasks  is  done  by  9  (Select 
ABDR  maintenance)  and  201  (Select  Maintenance  Level).  The  travel  time  takes  place  in  201.1 
(Travel  to  On-Equipment),  201.2  (Travel  to  Off-Equipment),  201.3  (Travel  to  Depot),  and  211 
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(Travel  to  system).  Maintenance  is  completed  at  202  (Do  On-Equipment  Maintenance),  203  (Do 
Off-Eqxiipment  Maintenance),  204  (Do  Depot  Maintenance)  and  212  (Do  ABDR  Maintenance). 

Task  9  -  Select  ABDR  Maintenance  -  -  This  task  determines  if  the  ABDR  team  is 
appropriate  for  the  repairs  for  this  system  and,  if  so,  sends  the  ABDR  team  to  the  system. 
Otherwise,  the  system  is  sent  to  the  other  three  maintenance  levels.  This  is  a  zero  time  task.  If  the 
number  of  systems  waiting  for  the  ABDR  team  is  less  than  the  user-defined  maximum,  and  all  the 
repairs  needed  for  this  system  are  repairable  by  the  ABDR  team,  then  the  system  waits  for  the 
ABDR  team.  If  the  system  is  waiting  for  the  ABDR  team,  21 1  (Travel  to  System  Location)  is 
initiated.  Otherwise,  the  system  is  sent  to  201  (Select  Maintenance  Level). 

Task  211  -  Travel  to  System  Location -- This  task  determines  the  time  the  ABDR 
team  will  take  to  get  to  the  system  based  on  travel  time  and  waiting  for  any  necessary  parts.5 

Task  201  -  Select  Maintenance  Level  -  -  Repair  tasks  are  routed  to  their  ap¬ 
propriate  maintenance  levels.  Since  the  system  can  only  be  in  one  place  at  any  one  time,  level  one 
repairs  must  complete  before  level  two  repairs  begin  and  level  two  repairs  must  complete  before 
level  three  repairs  begin.6  This  is  a  zero  time  task. 

Tasks  201.1,  201.2,  201.3  -  Travel  to  Maintenance  Level  -  -  Determine  the 
time  the  system  will  take  to  get  to  the  maintenance  level  based  on  travel  time  and  waiting  for  any 
necessary  parts. 

Tasks  202,  203,  204  •  Do  Maintenance  -  -  Tasks  are  released  into  maintenance 
based  on  a  priority  scheme  that  favors  the  system  with  the  least  cumulative  repair  hours  needed. 
When  all  the  repairs  on  a  system  are  complete,  the  system  returns  to  mission  ready  status. 

Each  maintenance  task  requires  a  specific  number  of  airmen  assigned  to  specific  AFSCs,  all  of 
them  working  for  the  time  set  forth  in  the  MTTR.  Before  we  begin  a  task,  we  check  to  ensure  that 

5  The  scenarios  that  were  developed  and  validated  for  the  Air  Force  overrode  this  feature  by 
specifying  zero  travel  times  between  maintenance  levels. 

6  The  Air  Force  version  assigns  On-Equipment,  Off-Equipment,  and  Depot  maintenance  to 
the  three  separate  levels.  For  that  reason,  it  is  not  strictly  true  that  level  1  and  level  2  maintenance 
cannot  be  p^ormed  concurrently.  This  is  discussed  further  in  the  "Limitations  of  this  Version" 
section  of  this  paper. 


the  manpower  pool  has  at  least  a  portion  of  the  required  resources  left  in  the  current  shift. 
Maintenance  cannot  begin  unless  there  are  manhours  available  in  the  necessar>  AFSCs. 

Maintenance  tasks  take  as  long  as  either  the  MTTR  or  the  number  of  manhours  left  in  the 
current  shift  for  the  needed  AFSCs,  whichever  is  smaller.  If  two  different  AFSCs  are  required  to 
perform  the  task,  they  are  each  checked  to  ensure  there  are  enough  manhours  left  in  the  current 
shift  to  perform  the  maintenance.  If  there  are  not  enough  manhours  left,  the  task  uses  the  available 
manhours,  then  waits  until  the  next  shift  (and  the  required  remaining  manhours  are  available) 
before  it  is  finished 

AF-MANCAP  also  calculates  the  maximum  number  of  AFSCs  us  d  and  ensures  that  the 
number  of  airmen  that  are  busy  at  any  one  time  does  not  exceed  the  crew  sizt  for  that  AFSC. 

When  any  maintenance  action  is  completed,  the  queue  of  tasks  awaiting  for  maintenance 
is  re-sorted  to  ensure  that  the  shortest  task  that  uses  the  available  combination  of  AFSCs, 
OTgaitization  level,  and  maintenance  hours  is  placed  in  maintenance. 

Finally,  data  are  recorded  in  this  task.  It  sets  variables  that  are  recordec  into  an  ASCII 
data  file  (system  identification  number,  repair  time,  AFSCs,  task  identification  number). 
Manpower  measures  and  daily  achieved  and  inherent  availability  are  post-processed  from  this  file 
following  the  termination  of  the  model. 

If  the  task  was  not  fully  completed,  it  is  returned  to  the  queue  in  front  of  the  maintenance 
level  to  wait  for  manhours  to  become  available.  Otherwise,  if  the  repair  was  online  and  this  was 
the  last  repair  task  scheduled  for  a  system,  the  system  is  returned  to  302  (Perform  Mission). 

AF-MANCAP  Outputs  -  -After  executing  the  simulation,  AF-MANCAP  produces  a 
number  of  reports  that  include  the  amount  of  maintenance  that  was  performed  by  each  type  of 
maintainer  and  the  resulting  reliability,  availability,  and  maintainability  of  the  systems  modeled. 
AF-MANCAP  also  includes  a  limited  capability  to  estimate  supply/support  requirements.^ 


7  This  ctqpability  was  not  validated.  The  current  version  includes  tables  of  Anny  data  for 
truck  capacities,  etc.  In  order  to  implement  this  capability  for  the  Air  Force,  new  data  tables  would 
need  to  be  creat^ 
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The  AF-MANCAP  oatput  consists  of  several  reports  that  compile  and  summarize  the 
results  of  the  simulation  run.  These  reports  are  summarized  below. 

Execution  Summary  -  The  execution  summary  lists  the  number  of  total  direct  and  indirect 
maintenance  manhours  at  each  organizational  level  throughout  the  simulation.  In 
addition,  it  specifies  the  total  number  of  maintenance  tasks  and  the  total  number  of  offline 
maintenance  tasks  that  were  performed.  The  execution  summary  also  lists  the  maximum 
maintenance  headcount  used  at  each  organizational  level. 

Maintenance  Headcount  Requirements  -  The  Maintenance  Headcount  Requirements 
Report  consists  of  a  number  of  frequency  histograms  that  specify  how  often  different 
crew  sizes  were  used  to  perform  maintenance  at  each  maintenance  level,  for  each  AFSC. 

Maintenance  AFSC  Requirements  -  This  report  specifies  the  total  number  of  maintenance 
hours  required  for  each  AFSC  at  each  organizational  level.  It  also  contains  the  maximum 
number  of  airmen  needed  for  that  AFSC  and  organizational  level  for  the  simulated 
period. 

Maintenance  Manpower  Requirements  -  This  provides  an  exhaustive  repon  of  each 
maintenance  action  that  could  have  happened  to  the  systems  being  simulated.  In 
addition,  this  report  lists  the  number  of  times  that  maintenance  action  was  performed,  and 
the  total  maintenance  manhours  spent  pofenming  that  action.  This  report  can  be  sorted  in 
order  to  enable  the  user  to  reorder  the  information  in  the  report.  The  report  can  be  sorted 
by  subsystem,  component,  maintenance  task,  AFSC,  or  organizational  level. 

Maintenance  Manhour  Requirements  Daily  Ranges  -  This  repon  contains  the  total  number 
of  maintenance  manhours  required  by  AFSC.  In  addition,  it  contains  the  average  number 
of  manhours  used  per  day  as  well  as  a  standard  deviation. 

.  laintenance  Workload  -  The  maintenance  workload  report  contains  a  calculation  of  the 
percent  utilization  of  each  AFSC.  It  calculates  this  utilization  by  dividing  the  number  of 
maintenance  manhours  required  by  the  maintenance  hours  available  for  each  AFSC. 

System  Availabilitv  -  This  report  contains  four  measures  of  system  availability,  calculated 
and  reported  on  a  daily  basis.  The  four  measures  of  availability  are  inherent,  achieved, 
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operational^,  and  sortie.  Inherent  availability  is  calculated  by  dividing  the  operational 
time  by  the  sum  of  the  operational  time  and  the  unscheduled  maintenance  time.  Achieved 
availability  is  calculated  by  dividing  the  operational  time  by  the  sum  of  operational  time, 
scheduled  maintenance  time,  and  unscheduled  maintenance  time. 

System  Maintainability  -  The  system  maintainal^ity  report  contains  the  maintenance  ratio 
(maintentmce  manhours  per  operational  hour)  for  each  subsystem.  In  AF-MANCAP, 
c^)erational  hours  are  defined  as  the  total  time  the  system  was  operationally  ready  (either 
performing  a  mission  or  out  of  maintenance  and  waiting  to  begin  a  mission).  This  report 
is  useful  for  identifying  the  "high  driver"  subsystems  of  your  system  (i.e.,  those 
subsystems  that  require  more  maintenance  than  the  others). 

Limitations  of  the  Version 

The  current  version  of  AF-MANCIAP  (Version  2.6)  was  developed  for  the  purpose  of 
demonstrating  whether  AF-MANCAP  provided  a  viable  method  of  estimating  manpower 
requirements.  Since  this  version  was  developed  directly  from  the  Army's  version  with  a  minimum 
effort,  there  are  several  areas  in  whicn  the  functionality  does  not  adequately  represent  the  manner  m 
which  Air  Fence  maintenance  organizations  operate.  These  areas  are  discussed  below. 

Launch  windows  -  The  current  version  of  AF-MANCZAP  sends  all  the  systems  out  to 
perform  a  scheduled  mission.  Then,  they  return  to  the  base  together.  While  this 
represents  the  way  in  which  most  Army  systems  operate  as  a  unit,  it  is  not  representative 
of  Air  Force  sortie  generation.  The  Air  Force  typically  launches  "cells"  of  aircraft  (e.g., 
four  aircraft  launch,  then  IS  minutes  later  another  four  aircraft  launch),  thereby  resulting 
in  staggered  requirements  for  launch,  recovery,  and  maintenance  tasks.9 


^  Operaticmal  availability  is  calculated  by  dividing  operational  time  by  the  sum  of  operation 
time,  all  maintenance  time,  indirect  maintenance  time  administrative  logistics  down  time 
(including  travel  to  and  from  maintenance  organizations,  waiting  for  spare  parts,  etc).  The  sortie 
availability  is  the  percentage  of  systems  that  completed  maintenance  and  were  operationally  ready 
when  a  mission  was  scheduled. 


9  Due  to  this  inconsistency,  AF-MANCAP  will  tend  to  overestimate  the  crew  sizes  required  to 
maintain  the  systems,  since  the  peak  manning  requirements  are  not  accurately  modeled.  However, 
the  AF-MANCAP  prediction  for  maintenance  manhours  are  not  affected  by  this  problem. 
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Mean  time  to  repair  variabilitv  -  AF-JvlANCAP  does  not  currendy  model  any  variability  in 
the  performance  times  for  each  maintenance  action.  The  Army  did  not  implement  this 
feature  due  to  a  lack  of  reliable  historical  data.  While  it  would  require  a  fairly  minor 
change  to  AF-MANCAP  to  implement  this  variability  as  well  as  a  selective  distribution 
parameter,  it  is  not  implemented  in  this  version.  With  fairly  long  rans  of  AF-MANCAP, 
the  effect  of  this  limitation  on  fidelity  will  be  fairly  minor,  particularly  with  respect  to  the 
output  of  maintenance  manhours.  It  is  more  likely  to  produce  an  effect  on  the  crew  size 
output. 

Off-line  maintenance  processing  -  The  current  version  of  AF-MANCAP  assumes  that  all 
maintenance  assigned  to  either  the  off-equipment  or  depot  organizational  levels  will  be 
performed  "off-line".  In  other  words,  the  system  will  not  be  held  in  maintenance  until 
the  action  is  complete.  Rather,  the  model  assumes  the  component  will  be  removed  from 
the  system,  replaced  with  a  spare,  and  the  system  will  be  returned  to  operational  status. 
NOTE;  This  implementation  of  off-line  cap«tbility  is  not  consistent  with  the  current 
implementation  in  the  Army  version  of  MANCAP. 

Indirect  maintenance  time  -  This  version  of  AF-MANCAP  coTtq>utes  indirect  roaintenance 
time  at  the  rate  of  2.5  hours  per  shift  for  each  maintainer.  This  calculated  value  is 
displayed  on  the  Execution  Summary  Report.  The  indirect  maintenance  time  does  not 
affect  the  model.  The  amount  of  indirect  maintenance  time  does  not  affect  the  main¬ 
tenance  manhour  report,  nor  does  it  affect  the  maintenance  headcount  reports. 

Spare  parts  processing  -  The  AF-MANCAP  tool  does  provide  for  the  unavailability  of 
spare  parts.  The  model  assumes  that  spare  parts  are  only  needed  for  'remove  &  replace' 
tasks.  Each  maintenance  organization  level  (i.e.,  on-equipment,  off-equipment,  depot) 
has  a  user-entered  probability  that  any  spare  part  will  be  available  when  it  is  needed.  If 
the  part  is  not  available,  the  length  of  the  delay  until  it  becomes  available  is  based  on  a 
series  of  rectangular  distributions. 

Runtime  -  AF-MANCAP  has  succeeded  in  that  it  provides  a  fairly  high  fidelity  unit-level 
maintenance  model  on  a  personal  computer.  The  user  interface  consists  of  pop-up 
windows  with  fairly  simplistic  data  entry  schemes.  Due  to  the  model's  power  and  the 
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limited  processing  environment,  model  execution  time  can  be  quite  long.  This  can  be 
improved  dramatically  with  a  math  co-processor. 


AF-MANCAP  Software  and  Data 

Software  Modules  -  -  AF-MANCAP  is  a  Disk  Operating  System  (DOS)  application. 
It  is  written  and  compiled  using  Microsoft  C.  It  consists  of  eighteen  separate  modules,  and  over 
60,000  lines  of  source  code.  The  executables  are  distributed  on  nine  5  1/4"  1.2  mega  byte  (MB) 
floppy  diskettes,  and  the  total  program  requires  approximately  5.5  MB  of  hard  disk  storage. 

Since  AF-MANCAP  is  hosted  on  a  640K  Random  Access  Memory  personal  computer,  it 
does  have  limitations  regarding  the  number  of  maintenance  actions  that  can  be  modeled.  This 
limitation  is  currently  300.  This  number  of  maintenance  actions  can  be  distributed  in  any  way 
among  up  to  300  subsystems.  NOTE:  A  maintenance  action  is  a  component  paired  with  a 
maintenance  task.  In  addition,  AF-MANCAP  models  are  limited  to  500  systems  and  180  days  of 
simulation. 

Utilities  -  -  AF-MANCAP  does  include  context-specific  help  screens.  These  help 
screens  also  include  access  to  a  glossary  of  terms  and  a  facility  through  which  the  user  can  examine 
data  sources  for  library  data. 

AF-MANCAP  does  include  an  extensive  set  of  utilities.  The  utility  programs  enable 
users  to  share  data,  generate  backups,  and  update  databases.  The  utilities  also  allow  the  user  to 
personalize  the  set-up  of  AF-MANCAP  to  access  different  storage  devices  (i.e.,  bemoulli  boxes). 

Data  Libraries  -  -  AF-MANCAP  contains  data  libraries  that  are  stored  in  a  relational 
database.  These  libraries  enable  users  to  "cut  and  paste”  from  existing  system  maintenance  data. 
This  can  significantly  decrease  the  amount  of  data  entry  required  to  perform  an  analysis. 

The  current  AF-MANCAP  libraries  contain  Air  Force  maintenance  data  that  are 
representative  of  a  notional  advanced  tactical  fighter  design.  The  data  were  developed  from 
Logistic  Suppon  Analysis  Records  and  Logistics  Composite  Model  (LCOM)  networks  associated 
with  the  F-15,  F-16,  and  F-18.  The  AF-MANCAP  libraries  also  contain  systems  data  from  22 
Army  systems  (e.g.,  MlAl  Tank,  Scout  Helicopter). 
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The  libraries  are  in  R:BASE  format,  and  were  designed  to  be  easy  to  update  as  more  data 
become  available. 


SUMMARY 

AF-MANCAP  is  a  desktop  tool  that  can  be  used  to  aid  decision  makers  in  assessing 
maintenance  requirements  for  new  or  proposed  systems.  The  tool  includes  embedded  stochastic 
simulation  models  that  increase  the  fidelity  of  the  maintenance  requirement  predictions.  The  tool  is 
fairly  easy  to  use,  with  help  screens  and  a  user’s  manual.  Further  information  can  be  found  in 
Archer  et  al.  (1990)  and  in  the  AF-MANCAP  Software  Documentation  technical  report. 
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GLOSSARY 

HARDMAN  III  -  HARDMAN  HI  is  a  collection  of  personnel  computer-based  software  tools  that 
were  developed  to  support  the  Army's  MANPRINT  initiative.  This  project  was  sponsored  by  the 
..\rmy  Research  Institute  in  Alexandria,  VA.  The  ARI  point  of  contact  is  Dr.  Laurel  Allender, 
(703)  274-9046. 

Maintenance  Action  -  A  maintenance  action  is  a  component  and  maintenance  task  pair.  Examples 
include:  inspect  engine,  troubleshoot  landing  gear. 

MOUBF  -  Mean  operational  unit  between  failure.  For  components  belonging  to  armaments 
subsystems,  this  is  measured  in  mean  rounds  between  failure.  For  components  belonging  to 
mt^lity  subsystems,  this  is  measured  in  mean  distance  between  failure.  For  all  other  components, 
the  measure  is  mean  time  between  failure.  For  unscheduled  maintenance  actions,  the  MOUBF  is 
interpreted  as  the  mean  of  an  exponential  distribution  and  failures  are  predicted  stochastically.  For 
scheduled  maintenance  actions,  the  MOUBF  is  applied  as  a  constant  failure  rate. 

MTTR  -  Mean  time  to  repair.  The  MTTR  is  modeled  as  a  constant  repair  time. 
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APPENDIX  B 

LCOM  EXPLAINEDi 


1  This  appendix  is  reproduced  from  Boyle,  E.  LCOM  explained.  (AFHRL- TP-90-58). 
Wright-Patterson  Air  Force  Base,  OH:  Logistics  and  Human  Factors  Division.  Reproduction 
approval  provided  16  December  1991. 
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APPENDIX  B:  LCOM  EXPLAINED 
SUMMARY 

This  paper  introduces  a  general  audience  to  the  Air  Forces ’s  Logistics  Composite  Model. 
LCOM  is  a  MonteCarlo  simulation  of  a  maintenance  organization  used  to  identify  optimal  base- 
level  resources.  An  important  LCOM  application  is  to  determine  maintenance  manpower 
requirements.  This  paper  describes  the  motives  and  some  of  the  processes  of  LCOM.  For  LCOM 
practitioners,  this  overview  is  not  need.  This  paper  is  included  in  the  Critical  Experiments  of 
Hardman  ITT  Utility  report,  for  lay  people,  who  wish  to  understand  the  basics  of  the  model  used 
for  the  convergent  validation  of  AF-MANCAP. 
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APPENDIX  B:  LCOM  EXPLAINED 


Introduction 

The  Logistics  Composite  Model  (LCOM)  was  created  in  the  late  1960's  through  a  joint  effort 
of  The  Rand  Corporation  and  the  Air  Force  Logistics  Command.  The  original  purpose  of  LCOM 
was  to  provide  a  policy  analysis  tool  to  relate  base-level  logistics  resources  with  each  other  and 
with  sortie  generating  cap-oility.  Logistics  resources  modeled  in  LCOM  include  maintenance 
people,  spare  parts,  and  aerospace  ground  equipment  (AGE).  LCOM  is  a  flexible  and  versatile 
model.  The  interaction  of  any  of  the  factors  can  be  studied  in  virtually  any  level  of  detail  the 
analyst  requires. 

Though  intended  to  examine  the  interaction  of  multiple  logistics  resource  factors,  LCOM’s 
most  important  use  has  been  in  establishing  maintenance  manpower  requirements.  A  large  porticm 
of  the  Air  Force  maintenance  workforce  is  justified  through  LCOM  simulation.  These  people  are 
said  to  be  "LCOM-eamed."  LCOM  is  connected  by  Air  Force  Regulation  25-7  to  the  manpower 
standards  process,  and  through  this  to  the  Air  Force  budget 

LCOM  software  documentation  is  abundant  (e.g.,  Drake  &  Wieland,  1982;  Aeronautical 
Systems  Division,  1990;  Air  Force  Manual  171-605).  And  several  LCOM  training  guides  have 
been  written  (e.g.,  Dengler,  1981;  Keller,  1977).  But  there  has  been  surprisingly  little  published 
focusing  on  the  LCOM  manpower  estimation  process  itself.  LCOM  modeling  is  often  cited  as  a 
basis  for  organizing  certain  kinds  of  manpower,  personnel,  and  training  (MPT)  analysis,  but  since 
few  people  understand  LOOM,  few  understand  why  this  connection  with  MPT  is  so  apt  Because 
of  its  complexity,  understanding  of  this  simulation  has  always  been  limited  to  a  very  small  group 
of  specialists.  This  essay  provides  a  concise  explanation  of  LCOM  for  the  non-specialist  who 
wants  to  understand  the  general  manpower  estimation  process  without  having  to  learn  LCOM's 
myriad  details.  The  objective  is  to  promote  LCOM  understanding,  not  LCXDM  mastery. 

LCOM  Simulation  Overview 

LCOM  simulates  the  work  of  a  maintenance  organization.  LCOM  smdy  objectives  may  differ 
widely,  but  the  usual  one  is  to  locate  the  best  -  or  optimal  -  mix  of  logistics  resources  to  support  a 
given  weapon  system  under  given  operating  conditions.  These  logistics  resources  can  be  spare 
parts,  support  equipment,  facilities,  or  human  resources  (i.e.,  maintenance  people).  An  LCOM 
simulation  is  analogous  to  an  experiment  in  which  variations  in  input  resources  are  related  to 
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variations  in  output.  In  LCX)M,  the  most  important  output  measure  is  usually  the  number  of  sorties 
flown.  In  manpower  studies  using  LCOM,  the  idea  is  to  find,  for  each  defined  Air  Force  Specialty 
(AFS),  the  lowest  manpower  input  that  just  achieves  the  desired  sortie  rate.  We  don’t  want 
manpower  to  be  too  high,  because  then  people  would  be  idle,  or,  in  LCOM  jargon,  underutilized. 
But  we  don't  want  manpower  to  be  too  low,  because  then  people  would  be  too  busy,  or 
overutilized.  We  would  lose  sorties  as  aircraft  needing  servicing  or  repair  wait  for  maintenance 
crews  to  become  available.  LCOM  simulation  for  manpower  announts  to  a  search  for  a  satisfactory 
balance  between  these  two  manpower  considerations  and  sortie  generation  potential. 

The  details  of  LCOM  modeling  may  seem  daunting  but  the  core  ideas  are  few  and  easy  to 
understand.  1  LCOM  can  be  thought  of  as  a  simple  counting  device.  The  simulation  logs  sorties 
and  other  performance  variables  from  manpower  levels  and  other  resource  information  supplied  to 
it  by  the  analyst.  From  this  perspective,  to  say  that  LCOM  "determines"  manpower  is  to  speak 
very  imprecisely.  In  fact,  the  analyst  supplies  the  manpower.  LCOM  simply  counts  the  sorties 
corresponding  to  that  manpower  level.  The  manpower  versus  sortie  trade-off  is  evaluated  as  a 
queuing  problem.  If  repair  waiting  lines  are  too  long,  more  servers  will  be  added  to  reduce 
waiting  times  and  thereby  meet  the  sortie  demand. 

The  analyst  describes  the  maintenance  environment  through  task  networks  and  resource 
definitions.  Air  Force  Specialties  (AFS),  with  corresponding  manpower  levels,  are  one  of  these 
resources.  These  can  be  described  in  any  manner,  but  they  must  be  declared.  The  analyst  also 
describes  the  number  of  aircraft,  mission  types,  spare  part  levels,  configuration  requirements,  and 
other  information.  These  data  are  used  as  input  to  the  simulation.  The  simulation  calls  for  aircraft 
with  specified  configurations  to  be  launched  at  particular  times.  If  aircraft  are  available,  they  are 
launched.  LCOM  forgets  about  them  after  they're  launched  but  remembers  them  when  they  return. 
If  they  are  "broke"  they  are  repaired.  If  they  are  not  "broke"  they  are  serviced  and  returned  to  a 
launch  pool.  LCOM  counts  and  summarizes  all  such  simulated  events.  The  rest  is  detail. 

Why  Simulation? 

The  Air  Force  has  long  favored  a  simulation  approach  to  aircraft  maintenance  manpower 
requirements.  The  main  reason  is  that  mathematical  work  measurement  methods,  which  arc  based 
on  expected  or  average  long  run  workload,  do  not  accurately  reflect  aircraft  maintenance  realities  or 
mission  imperatives  day  by  day.  The  volume  of  maintenance  work  fluctuates  over  time. 
Equipment  breaks  randomly,  and  peaks  in  sortie  generation  demand  may  arise  suddenly. 


*  The  person  who  quipped  ‘‘God  is  in  the  details”  was  surely  referring  to  LCOM. 
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Consequendy,  maintenance  work  and  maintenance  manpower  cannot  be  entirely  preprogrammed 
in  expectation  of  an  wderly  and  uniform  production  rate. 

Much  of  aircraft  maintenance  work  is  "unscheduled"  repair  of  equipment  that  breaks  in  a 
stochastic  -  or  random  -  manner.  Though  we  may  be  sure  that  aircraft  components  will  break  in 
the  long  run,  we  cannot  be  certain  when  they  will  break  in  the  short  run.  Hence,  to  man  work 
centers  according  to  the  long  run  average  workload  would  sometimes  mean  inadequate  sortie 
production  in  the  short  run.  A  simulation  approach  deals  with  random  variations  in  workload  by 
establishing  a  statistical  basis  for  estimating  the  sortie  risk  of  different  manpower  levels.  If 
randomness  in  maintenance  workload  and  spikes  in  sortie  demands  were  removed,  there  would  be 
little  reason  to  simulate.  A  deterministic  formula  or  other  “management  engineering”  approach 
might  be  used  instead.  In  LCOM,  manpower  is  wrapped  with  a  statistical  contidence  band. 

LCOM  is  called  a  Monte  Carlo  simulation  because  the  model  uses  random  draws  from 
equipment  failure  parameters  to  introduce  demands  for  unscheduled  maintenance  work.  Similar 
random  draws  determine  how  long  a  particular  repair  will  take.  The  analyst  specifies  the  mean, 
variance,  and  distribution  types  for  failures  rates  and  repair  times.  The  model  allows  chance  to 
play  a  role  in  failure  occurrences  in  any  given  simulation  trial.  As  a  consequence,  simulation 
trials  must  be  run  repeatedly  to  determine  the  "just  right"  manning  level  for  each  work  center. 
After  a  satisfactory  manning  level  is  found,  the  model  is  run  again  using  new  random  number 
seeds  to  determine  the  statistical  robusmess  of  a  given  manpower  level.  Variance  reduction  and 
other  techniques  can  make  the  simulation  process  more  efficient,  but  the  LCOM  iteration  process 
will  usually  be  more  time  consuming  than  a  deterministic  mathematical  approach  applied  to  the 
same  modeled  environment. 

The  interested  reader  will  find  illuminating  literature  on  military  manpower  requirements 
particularly  in  Rand's  work  in  the  late  1950's  and  early  1960's.  The  work  of  Houston  (1960, 
1962)  on  the  "personnel  subsystem"  concept  and  of  Levine  &  Rainey  (1959)  on  the  Base 
Maintenance  Operations  Model  describe  the  use  of  systems  analysis  tools  much  like  the  current 
LCOM  model  for  manpower  planning  to  suppon  Air  Force  systems.  The  technical  issues 
surrounding  maintenance  manpower  estimation  are  quite  old  and,  for  the  most  pan,  quite  well 
studied.  Newer  logistics  analysis  methods,  such  as  SAMSOM  (Bell  &  Stucker,  1971)  and  TSAR 
(Emerson  &  Wegner,  1985)  in  the  Air  Force,  and  manpower  tools  such  as  MANCAP  in  the  Army 
(Archer,  Griffith,  Laughery,  Maisano,  &  Kaplan,  1990),  attest  to  the  enduring  value  of  the 
simulation  approach  to  logistics  trade-off  analysis.  See  also  an  early  description  of  LCOM  by 
Fisher,  Drake,  Delfausse,  Clark,  and  Buchanan  (1968). 
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LCOM  Model  Description 


A  simplified  view  of  how  LCOM  can  model  the  aircraft  maintenance  world  is  shown  in  Figure 
B-1.  Aircraft  are  flown,  serviced,  repaired,  and  returned  to  flying  status  according  to  rules  defined 
by  the  analyst.  The  aircraft  are  processed  through  task  networks  that  describe  what  the  work  is 
and  what  it  requires.  For  this  reason  LCOM  is  also  called  a  network  processing  model. 

Maintenance  resource  levels  in  Figure  B-1  (i.e.,  spare  parts,  people,  and  equipment)  are 
defined  by  the  analyst,  not  by  LCOM.  The  model  will  call  upon  these  resources,  human  and 
otherwise,  in  supplying  aircraft  to  meet  the  flying  demand.  Generally  speaking,  if  too  few 
resources  are  provided,  the  aircraft  will  wait  Missions  will  be  cancelled  as  maintenance  queues  or 
backlogs  prevent  aircraft  from  flying.  If  too  many  resources  are  provided,  they  will  be  under¬ 
utilized;  in  effect,  wasted.  The  statistics  gathered  by  the  LCOM  simulation  provide  clues  about 
how  the  resource  levels  should  be  changed  to  improve  either  resource  utilization  or  sortie 
generation  potential.  For  a  manpower  study,  this  usually  means  adding  manpower  to  reduce 
maintenance  waiting  time,  or  reducing  manpower  to  improve  human  resource  utilization. 
Normally,  many  simulation  runs  are  needed  to  measure  the  effects  of  these  adjustments. 
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Figure  1.  LCXDM  Simulation  Logic.  (Adapted  fromDengler,  1981) 
Figure  B-1.  LCXDM  Simulation  Logic.  (Adapted  from  Dengler,  1981) 


LCOM  Software 

The  overall  structure  of  the  LCOM  software,  which  is  written  primarily  in  Simscript  n.5,  is 
shown  in  Figure  B-2.  LCOM  consists  of  a  preprocessor  program  (Input  Module),  a  simulation 
program  (Main  Module),  and  Post  Processor  Modules.  In  addition,  a  number  of  supporting 
programs  are  available  to  aid  the  data  build-up  process  of  LCOM.  This  Data  Preparation 
Subsystem  extracts  and  formats  Air  Force  maintenance  infoimation  from  deployed  aircraft  to  help 
create  the  LCOM  task  networks. 
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The  various  LCOM  forms  (Table  B-1)  constitute  the  LCOM  data  base.  After  error  checking, 
an  LCOM  preprocessor  converts  the  data  into  two  files:  the  initialization  ("init",  in  LCOM  jargon) 
and  the  exogenous  events  (or  "exog")  files.  The  init  file  describes  the  maintenance  environment  to 
be  simulated  and  provides  starting  values  for  the  prescribed  variables.  The  exog  file  contains 
flying  schedule  and  related  scenario  data  created  ft-om  the  mission  data  supplied  by  the  user.  This 
is  what  creates  demand  for  sorties  and  maintenance  work. 


Input  Module 


Main  Module 


Post  Processor 
Modules 


Output 
Reports 
(PSR) 


r 


Graphs, 
Diagrams, 
&  Reports 


Figure  B-2.  LCOM  Software  Structure  (adapted  fixtm  Dengler,  1981) 

The  Performance  Summary  Report  (PSR)  is  LCOM’s  principal  output.  Aeronautical 
Systems  Division’s  LCOM  Version  89 .D  lists  109  PSR  statistics  in  eight  categories: 


-  op^ations 

-  activities 

-  personnel 

-  supply 

-  chnn  Tcnair 

-AGE 

-  aircraft 
-facilities 


(e.g.,  sorties  flown) 

(e.g.,  average  time  to  get  resource) 

(e.g.,  manhours  used,  manhours  per  flying  hour) 
(e.g.,  number  of  items  back  ordered) 

(»•  g.,  number  of  items  repaired) 

(e.g.,  aerospace  ground  equipments  used) 

(e.g.,  number  of  aircraft  days  available) 

(e.g.,  facihties  used) 
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The  Post  Processor  Modules  produce  summary  statistics  for  the  entire  simulated  period.  These 
include  manpower  matrices  showing  demands  for  manpower  by  Air  Force  Specialty  (AFS)  by  time 
of  day,  and  usage  and  availability  of  spare  parts,  among  others.  The  manpower  matrix  and  parts 
reports  are  particularly  important  in  manpower  modeling  with  LCOM. 

The  simulated  work  environment  includes  scheduled  maintenance  needed  to  fuel,  arm, 
service,  and  inspect  aircraft.  This  is  described  in  the  main  servicing  network.  It  also  includes 
work  needed  to  fix  airplanes  that  have  broken  in  some  way.  This  is  described  in  the  unscheduled 
maintenance  network.  The  modeled  work  may  also  include  phase  (periodic)  maintenance.  Both 
organizational  (flightline)  and  intermediate  (shop)  tasks  are  described.  These  are  also  called  “on- 
equipment”  and  “off-equipment”  tasks,  respectively. 

The  analyst  may  define  so-called  maintenance  action  clocks  for  each  aircraft  subsystem, 
component,  or  part.  The  maintenance  action  clock  "decrement"  governs  the  rate  at  which 
equipment  fails,  and  this,  in  turn,  governs  the  volume  of  unscheduled  maintenance  manhours. 
Often,  these  clocks  are  set  in  mean  sorties  between  failures,  but  other  mcirics  can  be  used.  The 
reliability  of  equipment  is  estimated  from  maintenance  experience  with  fielded  systems  or  from 
engineering  data  for  new  systems. 

A  common  LCOM  modeling  scenario  is  to  cycle  aircraft  in  and  out  of  the  main  servicing 
network  until  a  maintenance  action  clock  has  breached,  whence  it  passes  the  aircraft  through  the 
unscheduled  maintenance  (repair)  networks  corresponding  to  the  failed  equipment  item.  When 
repaired,  the  aircraft  returns  to  a  mission-ready  pool  for  assignment. 

A  large  array  of  options  and  related  "instrumentation"  have  been  added  to  LCOM  over  the 
years.  These  allow  the  maintenance  environment  to  be  modeled  with  greater  detail,  flexibility,  and 
realism.  Even  the  PSR  can  be  tailored.  While  these  doubtless  make  LCOM  difficult  to  master, 
they  do  not  alter  the  model’s  “fly,  fix,  and  figure”  logic  in  any  fundamental  way.  They  do  make 
it  difficult  to  describe  LCOM  briefly  without  misleading  through  oversimplification. 

LCOM  Data  Base 


The  LCOM  input  forms  are  listed  in  Table  B-1 . 
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TABLE  B-1.  LCOM  INPUT  FORMS 


Form  Name  Purpose 

Task  Network  Every  task's  name,  sequence  node,  and  selection  mode. 

Task  Definitions  Every  task's  name,  time  (mean  &  variance)  and  resource  ID  and 

quantity  (AFS,  crew  size,  spare  part,  AGE) 

Resource  Definitions  AFS,  spare  parts,  aircraft,  AGE,  and  failure  clocks  are  identified 

Clock  Decrements  Equates  equipment  failure  probabilities  to  sorties 

Shift  Change  Policy  Defines  shift  length  and  how  resources  are  to  be  allocated  to  shifts 

Mission/Activity  Defines  resources  entering  the  network  and  the  required  aircraft 

Entry  Points  configuration  aUowing  tracking  and  assignment  of  aircraft  to  missions 

Priority  Specifications  Describes  how  to  handle  task  conflicts  when  using  resources  through 

preempting,  expediting,  and  restarting  rules 

Sortie  Generation  Data  Defines  mission  types  and  other  scenario  data 

Performance  Defines  PSR  reporting  structure 

Summary  Reports 

Statistical  Distributions  Specifies  distribution  types  (normal,  log  normal,  exponential,  etc.) 

Aircraft  Assignment  Defines  aircraft  external  and  internal  configuration  search 
Search  Patterns  selection  sequence 

Internal  Equipment  Defines  internal  equipment,  its  authorization,  and  the  network  location 

Authorization  (Ganges  effecting  its  quantities 

Internal  Equipment  Defines  internal  equipment  groupings  or  combinations  by  aircraft 

Group  Definitions 


Attribute  Definitions  Defines  an  input  format  for  combining  data  on  separate  LdDM  forms 


Notes:  ( 1 )  LCX)M  form  numbers  are  not  listed 

(2)  AFS  =  Air  Force  Specialty 

(3)  AGE  =  Aerospace  Ground  Equipment 


LCOM  Task  Language 
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In  LCOM,  most  maintenance  tasks  are  described  as  actions  taken  on  a  piece  of  hardware. 
These  tasks  require  resources  (people,  parts,  and  AGE)  and  time.  The  actions  applicable  to  people 
are: 


On-eauipment  (Flighdine) 


Qff-eqwpmfflt  (Shop) 


X  =  Access  (Use  AGE) 

T  =  Troubleshoot 

R  =  Renoove  and  replace 

H  =  Inspect 

M  =  Minor  repair 

V  =  Verify  system  works 

J  =  Aircraft  handling 

B  =  Loading/downloading  munitions 


L  =  Component  identification 
W  =  Check/repair  component 
K  =  Component  checlu  OK 
N  =  Check  and  condemn 
Y  =  Disassemble/reassemble 


When  these  action  codes  are  paired  with  equipment  Work  Unit  Codes,  a  concise  task 
descriptive  language  is  created.  For  example,  ''T74AB0"  in  LCOM  means  "troubleshoot  the 
(F-16)  radar  low  power  RF."  The  entire  LCOM  language  for  unscheduled  maintenance  is  spoken 
in  this  "action  taken/work  unit  code"  manner.  For  general  aircraft  servicing  tasks  and  other  work 
that  cannot  be  tied  precisely  to  specific  equipments,  words  like  FUEL,  LAUNCH,  and  TOW  are 
also  used. 

The  task  descriptive  vocabulary  used  by  LCOM  is  exact  but  it  is  also  rather  limited.  There  is 
no  implication  in  LCOM  maintenance  networks  of  what  military  psychologists  would  call  task 
analysis.  That  is,  the  only  things  LCOM  knows  about  a  task  is  who  does  it,  how  many  people  are 
needed,  who  may  substitute,  what  support  equipment  is  needed,  and  how  long  the  task  takes. 
Through  the  maintenance  action  clocks,  LCOM  also  knows  how  often  a  task  is  apt  to  occur.  It 
knows  nothing  else  about  the  qualitative  aspects  of  the  work.  Task  difficulty,  personnel  skills, 
safety  considerations,  and  so  on  are  not  considered  by  LCOM. 

LCOM  Task  Networks 

Maintenance  tasks  are  described  in  networks  that  define  their  logical  flow.  These  networks  can 
be  defined  in  many  different  ways  and  in  any  level  of  detail.  The  task  in  Figure  B-3,  for  example, 
begins  when  a  failure  clock  for  Part  X  has  breached.  The  network  section  applicable  to  Part  X  is 
then  activated.  The  aircraft  will  halt  processing  through  the  main  servicing  network  while 
maintenance  is  performed. 
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Figure  B-S.  LCOM  Network  Example 

The  diagram  shows  that  it  takes  a  crew  of  two  specialists  with  AFS  454X2  three  tenths  of  an 
hour  to  identify  and  access  the  problem.  A  repair  action  taking  .6  hours  will  result  70  percent  of 
the  time,  a  remove  and  replace  action  taking  .8  hours  30  percent  of  the  time.  After  a  check,  the 
aircraft  continues  processing  toward  mission-ready  status.  Shop  manhours  are  also  generated 
when  the  failed  part  arrives  for  repair.  The  frequency  with  which  this  network  section  is  activated 
is  governed  by  the  maintenance  action  clock  describing  the  equipment’s  expected  reliability.  The 
manhours  consumed  by  this  maintenance  over  the  simulated  period  are  summed.  Eventually,  these 
manhours  will  contribute  to  an  LCXDM  manpower  estimate. 

One  of  LCOM's  distinctive  features  is  the  wide  array  of  task  networking  controls  it  provides. 
These  can  be  used,  for  example,  to: 

-  "call"  other  tasks  or  networks, 

-  create  probabilistic  branching  (Figure  3) 

-  skip  over  or  accomplish  tasiu  in  parts 

-  de^e  sequential  and  paraUel  task  strings 

-  consume  and  generate  parts 

-  change  the  location  of  resources 

-  decrement  failure  clocks 

-  model  parts  cannibalization  (i.e.,  taking  parts  from  another  aircraft) 

An  LCOM  data  base  (i.e.,  the  assembled  forms)  can  run  to  several  thousand  lines  of  code  for  a 
detailed  weapon  system  study.  The  bulk  of  this  code  consists  of  task  networks,  resource 
definitions,  and  task  defrnitions.  A  special  input  coding  device,  the  Extended  Form  11,  can  be 
used  to  consolidate  the  information  contained  on  separate  L(X)M  forms. 
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Deriving  Maintenance  Manpower  With  LCOM 


LCXDM  models  are  typically  run  for  debugging  purposes  with  resources  unconstrained.  This 
means  that  essentially  unlimited  quantities  of  people,  parts,  and  equipment  are  made  available. 
Initial  wide-open  simulation  permits  the  analyst  to  confirm  that  sorties  are  being  demanded,  that 
maintenance  is  occurring,  and  that  the  modeled  environment  conforms  with  the  data  base  and 
operational  logic  prescribed  for  it.  An  unconstrained  LCOM  simulation  can  be  used  to  determine 
the  maximum  achieveable  theoretical  performance. 

But,  typically,  we  don't  want  the  theoretical  maximum  sortie  rate.  We  want  a  sortie  rate  that 
reflects  real-world  flying  requirements.  And,  in  any  case,  we  don't  have  unlimited  resources  to 
work  with.2  Hence,  resource  optimization  in  LCOM  is  a  process  of  systematically  adjusting 
resources  until  the  LCOM  sortie  rate  settles  around  the  desired  sortie  rate  and  other  criteria,  such  as 
manpower  utilization  limits,  are  not  violated.  This  process  is  called  constraining.  It  is  through 
constraining  that  the  analyst  eventually  finds  the  "just  right"  manpower  level  for  each  work  center. 

Since  resources  interact  in  complex  ways,  it  is  common  to  constrain  their  levels  one  at  a  time. 
For  example,  it  would  make  little  sense  to  try  to  optimize  manpower  if  scarcity  of  spare  parts  and 
equipment  prevented  people  from  doing  work.  Hence,  in  manpower  studies,  attention  falls  first  on 
constraining  spare  parts  and  equipment  down  to  levels  which,  upon  simulation,  restrict 
performance  to  some  pre-defined  criterion. 

Often,  this  criterion  is  the  "Not  Mission  Capable  -  Supply"  (or  NMCS)  rate,  a  statistic 
produced  by  the  PSR.  If  the  LCOM  scenario  specifies  a  NMCS  rate  of,  say,  10  percent,  the 
objective  of  parts  constraining  simulation  trials  is  to  establish  parts  levels  that  produce  an  average 
aircraft  availability  factor  of  90  percent  The  Post  Processor  Modules  can  produce  reports  useful 
for  finding  appropriate  spare  parts  levels.  Similar  procedures  can  be  used  for  equipment 
constrairdng,  though  AGE  is  less  often  at  issue  in  LCXDM  manpo'^'er  studies. 


^  Of  course,  without  resource  scarcity  there  would  be  no  'onomics.  And  LCOM  would  be  superfluous. 
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Manpower  Factors 


The  relationship  of  manpower  factors  to  sortie  rate  is  shown  in  Figure  B-4. 


Figure  4.  Manning  Factors  (Adapted  from  Dengler,  1981) 

During  manpower  constraining,  the  LCOM  analyst  must  consider  which  of  these  factors  is 
driving  the  manpower  requirement  for  each  AFS.  Other  things  equal,  the  sortie  rate  will  govern 
the  manpower  factors.  These  are: 

Post  Manpower:  Crews  dedicated  to  a  fixed  post  (e.g.,  end  of  runway  checks)  for  a  fixed 
period  and  who  cannot  be  reassigned  during  the  work  shift. 

Crew  Size:  Each  LCOM  task  has  a  defined  minimum  crew  size.  Imagine  an  AFS  with  20 
tasks  in  all,  19  of  which  require  two  people,  and  one  of  which  requires  three  people.  A  charming 
LCOM  locution  identifies  this  latter  task  as  the  "maximum  minimum  crew  size."  As  a  general  rule, 
manpower  on  each  shift  should  not  be  lower  than  this  number. 

Direct  Labor:  The  manpower  level  needed  to  accomplish  the  direct  manhours  of  work 
generated  by  the  simulation.  It  is  shown  as  a  near  linear  increasing  function  of  sorties  flown. 

Peak  Demand:  Sortie  demand  may  have  an  irregular  pattern  through  the  day.  Massed  fights  or 
surge  conditions  may  require  many  people  to  be  working  at  the  same  time  for  brief  periods.  More 
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people  may  be  needed  to  cover  these  peak  demands  than  might  be  provided  by  applying  the  other 
maniing  factors  alone. 


Manpower  Constraining 

When  spare  parts  constraining  is  done,  manpower  constraining  starts.  The  required  manning 
levels  for  each  work  center  (or  AFS)  are  determined  through  a  progressive  and  systematic  process 
of  manpower  constraining  over  many  simulation  runs.  This  process  of  allocating  and  reallocating 
manpower  calls  upon  LCOM  statistical  reports  as  well  as  the  anai>  st’s  judgment. 

Dengler  (1981)  describes  the  following  method.  In  the  equation, 

AFS  Manhours  Used  (from  PSR  Statistic  29) 

M  (s)  =  - - ^ - 

(Utilization  Factor)  x  (Number  of  Days)  x  (Shift  Length) 

manhours  used  by  each  AFS  are  converted  to  average  daily  number  of  people  required  for  a  shift 
[M  (s)]  by  taking  shift  length,  days  simulated,  and  manpower  utilization  or  availability  factors  into 
account.  Utilization  factors  are  specified  as  the  percent  of  available  manhours  that  can  be  allocated 
for  direct  work.  The  upper  limits  vary  by  AFS,  but  average  about  80  percent.  'Tie  availability 
factors,  by  current  Air  Force  policy,  are  144,5  hours  per  person  per  month  in  peacetime  and  244 
hours  per  uionth  in  v/artime.  The  analyst  must  decide  which  policy  is  applicable  to  his  simulation 
problem.  The  shift  manning  levels  so  derived  become  starting  values  for  manpower  constraining 
runs.  AFS  manning  should  not  be  lower  than  the  maximum  minimum  crew  size  if  no  AFS 
substitution  rules  have  been  defined. 

LCOM  simulations  are  performed  using  so-called  "change  cards"  which  list  authorized 
resource  levels  for  the  run.  The  number  of  days  to  be  simulated,  the  simulation  run  time,  must  be 
large  enough  to  ensure  that  random  effects  do  not  unduly  disiort  the  overall  results.  'The  sortie  rate 
target  and  the  number  of  PAA  (Primary  Authorized  Aircraft)  for  tiie  simulation  must  be  taken  into 
account.  Dengler  (1981)  recommends  1 12  days  for  a  24-^A  \  unit  and  38  days  for  a  72-PAA  unit; 
both  yield  2,000  sorties. 

The  analyst  is  guided  in  setting  manning  levels  for  subsequen*^  LCOM  runs  by  monitoring  the 
sortie  rate  and  manpower  utilization  statistics  associated  with  a  given  manning  level.  AFSs  that 
may  need  additional  maiipower  can  often  be  identified  by  examining  Lhe  Manpower  Matrix  Post 
Processor,  which  shows  AFS  "backorde  r"  statistics.  Ihe  analyst  must  determine  whether  repair 
delays  in  particular  work  centers  are  constraining  the  sortie  rate.  Such  delays  might  be  tolerated  if 
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they  are  not.  Work  center  manning  might  be  reduced  if  the  average  utilization  rate  falls  below 
established  standards. 

Finally,  the  actual  manpower  -  the  bottom  line.  After  the  analyst  has  completed  all  AFS 
manning  adjustments  and  satisfied  himself  through  confirmatory  LCOM  runs  that  he  has  reached 
the  optimal  manpower  levels  for  each  AFS,  he  has  one  final  calculation  to  make.  The  number  of 
authorizations  (i.e.,  the  number  of  whole  people  to  be  listed  on  manning  documents)  for  each  AFS 
depends  on  the  total  daily  LCOM  requirement  for  all  shifts,  the  monthly  manpower  availability 
factor,  the  work  days  per  month,  and  the  shift  length.  The  equation  below  shows  how  this 
calculation  is  made. 

Average  Daily 

LCOM  Derived  Work  Days  Shift 

Manpower  x  Per  Month  x  Length 

M  = - 

Manpower  Availability  Factor 

The  term  "whole  people"  above  is  used  advisedly.  Divisic”  with  fractional  availability  factors 
(e.g.,  244.8  hours  per  month  for  wartime)  will  produce  fractiona  anpower  requirements.  Since 
we  can  authorize  people  only  in  whole  (integer)  units,  tables  for  rounding  these  fractions  into 
whole-person  equivalents  have  been  developed. 

Certain  Matters 

Manpower  Availability  and  Utilization.  Availability  is  the  number  of  hours  per  month  a  person 
can  be  allocated  to  a  duty  post.  For  peacetime,  144.5  hours  is  used;  for  wartime,  244.8  hours. 
Utilization  is  the  percentage  of  a  person's  duty  time  that  can  be  allocated  to  direct  work.  There  are 
published  standards  for  utilization.  Manpower  requirements  computed  from  LCOM  -  and  from 
other  methods  -  take  both  factors  in  account  since  both  influence  manpower  requirements. 

AFS  Task  Inventory.  LCOM  describes  only  direct  maintenance  work.  The  indirect  work 
maintenance  people  do  is  accounted  for  through  the  manpower  utilization  factors,  but  the  work 
itself  is  not  described.  Hence,  LCOM  data  bases  will  not  normally  contain  a  complete  inventory  of 
the  work  of  each  specialty. 

LCOM  vs.  Standard  Manning.  In  general,  only  those  work  centers  whose  manning  levels 
directly  constrain  sordes  are  modeled  in  LCOM.  Shop  overhead,  maintenance  management,  and 
certain  off-equipment  AFSs  are  excluded.  Depot  manpower  is  likewise  excluded.  About  half  of 
unit-level  maintenance  manpower  is  derived  with  LCOM  across  the  Air  Force.  (See  Furry, 
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Bloomberg,  Lu,  Roach,  &  Schank,  1979.)  The  rest  of  the  manpower  requirement  is  determined 
by  application  of  manpower  standards  or  by  other  means. 

LCOM  Data  Base  Support.  LCOM  data  bases  are  created  in  part  from  the  Air  Force 
Maintenance  Data  Collection  System  (MDC).  A  number  of  computer  programs  have  been  created 
to  process  these  data  into  LCOM  format.  These  programs  are  not  actually  part  of  LCOM  but  they 
are  important  for  LCOM  modeling.  The  Common  Data  Extraction  Programs  (CDEP)  and  other 
software  aids  -  the  Data  Preparation  Subsystem  -  could  be  called  LCOM's  front  porch. 

LCOM  Audit.  A  so-called  "Operational  Audit"  is  conducted  to  verify  the  simulated 
maintenance  world.  LCOM  technicians  will  visit  operating  bases  for  the  weapon  system  under 
study  to  check  the  accuracy  of  MDC  data,  verify  AFS-iask  assignments  and  crew  sizes,  determine 
maintenance  procedures  and  task  times,  and  so  on. 

LCOM  Software  Vintage.  LCOM  is  basically  a  1960's  style  "batch"  system.  It  is  not  very 
user-friendly,  not  user-interactive.  Until  recently,  LCOM  was  confined  to  mainframe  computers. 
Many  Air  Force  users  now  run  LCOM  on  IBM  9370  S’lper  mini's.  Some  run  LCOM  on  VAX 
1 1/780  series  machines.  We  are  not  likely  to  see  LCOM  software  running  on  personal  computers 
any  time  soon,  although  proposals  for  rewriting,  compressing,  and  updating  the  LCOM  code  to 
make  this  possible  are  sometimes  heard.  See  the  "LCOM  2000"  study  (Dymond,  et  al.,  1987)  for 
proposed  enhancements  to  the  LCOM  software. 

LCOM  Substitutes.  The  basic  queueing  processes  and  simulation  logic  of  LCOM  can  be  easily 
replicated  by  any  number  of  competing  methods.  SLAM  (Simulation  Language  for  Alternative 
Modeling)  and  Micro-SAINT  are  well  known  examples.  The  Army  Research  Institute's 
MANCAP  (Manpower  C^apabilities  Predictor),  one  of  the  new  HARDMAN  IQ  tools,  was  inspired 
by  LCOM.  But  LCOM  remains  unique.  Its  flexibility,  detail,  range  of  application,  and  data  base 
support  far  exceed  those  of  any  potential  substitute  within  its  domain.  For  a  given  study, 
equivalence  of  model  criteria  might  mean  equivalence  in  model  credibility,  but  LCOM  findings  still 
tend  to  be  used  as  the  standard  for  comparing  manpower  results. 

An  LCOM  Sampler 

The  LCOM  process  lends  itself  to  innovative  applications.  LCOM  has  been  used  with  other 
models  and  it  has  been  extended  to  systems  other  than  aircraft  and  to  the  other  Services.  The 
applications  discussed  below  convey  some  idea  of  LCOM’s  use  within  the  MPT  domain. 
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LCOM  in  Acquisition.  LCOM  has  been  paired  with  comparability  analysis  to  produce  early 
estimates  of  maintenance  manpower  for  new  systems.  The  work  of  Tetmeyer  (1974)  and  his 
colleagues  at  the  Air  Force  Human  Resources  Laboratory  and  at  Aeronautical  Systems  Division  is 
the  best  known  example.  The  comparability  approach  pioneered  by  Tetmeyer  is  now  prescribed  by 
Logistics  Support  Analysis  (MIL-STD  1388-lA).  The  basic  idea  is  to  create  a  baseline  equipment 
configuration  for  a  new  system  by  using  subject-  matter  experts  to  identify  existing  systems  that 
are  most  like  the  projected  new  system.  Tetmeyer’s  approach  emphasizes  the  development  of 
equipment  reliability  "deltas"  which  are  used  to  adjust  LCOM  failure  clocks.  The  notion  of 
baseline  comparison  systems  so  prominent  in  MPT  analysis  for  new  systems  is  rooted  in  this 
LCOM-oriented  work.  (See  also  Maher  &  York,  1974;  Tetmeyer  &  Moody,  1974;  and  Tetmeyer, 
Nichols  &  Deem,  1976) 

The  "Skill  Level  Problem."  LCOM  modeling  assumes  that  all  people  within  an  AFS  will 
perform  a  task  in  the  same  way.  Every  person  is  assumed  to  be  task  qualified  and  to  take  the  same 
amount  of  time  to  do  a  task.  Howell  (1981)  showed  what  could  happen  if  "three-levels" 
(inexperienced  people)  predomina  i  the  workforce.  He  adjusted  the  LCOM  task  times  using 
subject-matter  expert  judgments  comparing  "three-levels"  against  "five-levels"  (experienced 
people).  LCX)M  projected  much  larger  manpower  requirements  with  the  "three-level"  workforce 
since  inexperienced  people  were  judged  to  require  more  time  to  do  the  same  work  as  "five-level" 
technicians.  Garcia  &  Racher  (1981)  attempted  to  incorporate  Air  Force  occupational  survey  data 
on  task  difficulty  and  time  spent  on  maintenance  tasks  identified  in  LCOM  and  obtained  similar 
results. 

MPT  Integration.  The  LCOM  process  has  been  expanded  by  the  SUMMA  (Small  Unit 
Maintenance  Manpower  Analyses)  model  to  serve  as  an  integrating  mechanism  for  manpower, 
personnel,  and  training  analysis.  The  SUMMA  model  (Boyle,  1990)  uses  LCOM  task  data 
supplemented  with  subject-matter  expert  judgment  to  identify  improved  AFS-task  alignments.  The 
objective  is  to  limit  manpower  requirements,  especially  for  small  unit  deployments  by,  in  effect, 
enlarging  maintenance  jobs.  An  MPT  projection  model  informs  the  analyst  of  potential  aptitude, 
training,  and  cost  impacts  of  job  merger  options,  and  an  analytic  manpower  forecast  is  provided. 
LCX)M  upload  and  download  software  is  also  included  in  the  SUMMA  package.  The  SUMMA 
model  attempts  to  tie  MPT  analysis  to  altered  AFS  policies  and  to  tie  these,  in  turn,  to  LCOM. 
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Critical  Experiments  of  HARDMAN  III  Utility 

for  Use  by 
Air  Force  Analysts 


1.  INTRODUCTION: 

This  project  addresses  the  utility  of  a  suite  of  analysis  tools  for  evaluating  the 
manpower,  personnel,  and  training  (MPT)  impacts  and  trade-off'  of  emerging 
Air  Force  weapon  systems.  The  technologies  addressed  in  the  software  products 
are  based  on  components  of  the  HARDMAN  Ill.n  analysis  tools: 
Manpower-Based  System  Evaluation  (MAN-SEVAL)  and  Manpower 
Capabilities  Model  (MANCAP  II).  During  the  summer  of  1991  the  original 
HARDMAN  III.II  software  products  were  modified  for  Air  Force  use.  The  new 
tool  was  called  AF-MANCAP.  Also,  a  test  library  of  components  from  the  F-18, 
F-15  and  F-16  was  created  from  historical  Logistic  Support  Analysis  Records 
and  Logistics  Composite  Model  networks.  The  prototype  software  products 
were  developed  by  HAY  Systems,  Inc.,  Washington  D.C.  and  Micro  Analysis 
and  Design,  Inc.,  Boulder,  Colorado  for  the  Air  Force  Armstrong  Laboratory, 
Brooks  Air  Force  Base,  Texas. 

2.  SCOPE  AND  FIELD  OF  APPLICATION: 

2.1  SCOPE: 

The  AF-MANCP  products  were  developed  to  obtain  user  reactions  to  the 
analytical  tools  and  manpower  analysis  process.  Your  assessment  will  provide 
information  for  the  determination  of  the  characteristics  and  requirements  for  a 
production  version  of  the  software  products. 

2.2  EVALUATION  AUDIENCE  AND  FIELD  OF  APPLICATION: 

Subject  matter  experts  (SME)  who  prepare  manpower,  personnel,  and  training 
inputs  for  Manpower  Estimate  Reports  (MER)  and  Integrated  Manpower, 
Personnel,  and  Comprehensive  Training  &  Safety  (IMPACTS)  Program  Plans. 

3.  REFERENCES: 

•  Statement  of  Work,  Critical  Experiments  of  HARDMAN  III  Utility  for  Use  by 
Air  Force  Analysts  F49642-88-D0()03,  Delivery  Orders  5023  and  5027. 

•  Acquisition  Guide  for  Measurement  (draft).  Rating  and  Assessment, 
International  Standards  Organization,  2  March  1991. 

4.  AIR  FORCE  POINT  OF  CONTACT: 

•  Dr.  Bruce  Gould,  AL/HRMM,  Brooks  AFB,  TX  78235-5601,  (512)  536-3648. 


i  Please  note  that  all  information  provided  is  considered  confidential  and 
will  only  be  used  by  the  development  team  for  the  purpose  of  improving 
i  the  software  products. 

AF-MANCAP  Product  Assessment  2 
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Background  Information 


1.  Your  job  title  is: _ 

2.  Your  Primary  AFSC  or  Occupational  Series  is:. 

3.  Your  Duty  AFSC  or  Occupational  Series  is: _ 


4.  Military 

If  military  your  grade  is; 

5.  Civilian 

If  civilian  your  grade  is-. 


[  ]  yes  [  ]  no 


and  your  TAFMS  is: _ 

[  ]  yes  [  ]  no 
and  your  Govt.  Service  Time  is: _ 


6.  Your  civilian  education  background  is: 

Highest  degree: _  field  of  study: _ 

7.  Have  you  developed;  (If  "yes"  is  checked  provide  program  name(s) 


a. 

Manpower  Estimate  Report? 

Programfsi? 

1  ]  yes  [  ]  no 

b. 

IMPACTS  Program  Plan? 

[  ]  yes  1  ]  no 

Programfsl? 

c. 

LCOM  Model? 

1  ]  yes  [  ]  no 

Program(s)? 

d. 

System  Training  Plan? 

[  ]  yes  [  ]  no 

F^gram(s)? 

e 

Predecessor/Comparability  Analysis? 

1  ]  yes  [  ]  no 

Program(s)? 

f. 

Baseline  Comparison  System? 

[  ]  yes  [  ]  no 

Programfsi? 

g.  Other:  Describe  any  other  examples  of  manpower,  personnel,  and  training 
analyses  that  you  may  have  accomplished? _ 


Do  you  have  Disk  Operating 

System  (DOS)  Experience?  t  1  1 

a.  If  "yes"  provide  the  years  of  experience: _ (years) 

b.  If  "yes"  list  programs  you  have  used  in  the  last  year: _ 
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I  Background  Information  (cont) 

9.  Have  you  used  any  of  the  following 

databases  or  tools  for  manpower,  personnel,  or  training  analyses: 


a.  Operational  Research  Data  Bank  (ORDB)  at 
Brooks  Air  Force  Base? 

b.  Lesson  Learned  database  at  Wright-Patterson  Air 
Force  Base? 

c.  Crosswalk  or  Footprint  databases  provided  by  the 
Training  and  Performance  Data  Center  (TPDC), 
Orlando,  Florida? 

d.  Logistic  Support  Analysis  Records  (LSAR)? 

e.  Logistics  Composite  Model  (LCOM)  Networks? 

f.  Training  System  for  Maintenance 
(TRANSFORM)? 

g.  Small  Unit  Maintenance  Manpower  Analysis 
(SUMMA)? 

h.  Manpower  Standards  Development  System 
(MSDS)? 

i.  Maintenance  Operational  Data  Access  System 
(MODAS)? 

j.  Core  Automated  Maintenance  System  (CAMS)? 

k.  Occupational  Survey  Repon  Data  from  Randolph 
AFB? 

l.  Cost  Oriented  Resource  Estimating  (CORE) 
model? 

m.  CREWCHIEF  model? 

n.  Other  databases  or  models? 


I  ]  yes  [  ]  no 


[  ]  yes  [  ]  no 


[  ]  yes  [  ]  no 

I  ]  yes  I  ]  no 
[  ]  yes  [  ]  no 


I  J  yes  [  ]  no 


[  ]  yes  [  ]  no 


[  1  yes  I  )  no 


I  ]  yes  [  ]  no 
[  )  yes  [  ]  no 


[  ]  yes  [  ]  no 


[  ]  yes  [  ]  no 

[  ]  yes  [  ]  no 
[  ]  yes  [  ]  no 
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Instructions 


This  part  of  the  survey  contains  questions  on  five  characteristics  of  software 
products.  The  characteristics  are:  functionality,  reliability,  usability, 
efficiency,  and  maintainability.  For  each  of  the  characteristics  select  one 
response  (A,  B,  C,  or  D  by  marking  the  [  ]  that  represents  your  observations 
about  the  prototype  software  product.  Please  evaluate  each  characteristic. 
Please  provide  comments  for  any  "C"  or  "D"  ratings.  If  you  have  an 
questions  regarding  these  instructions  or  the  survey  please  ask  the  consultant. 


i  Expleuxation  of  Rating  Scales 

The  survey  questions  are  designed  to  assess  if  the  software  products  will 
fulfill  the  needs  of  a  manpower,  personnel,  and  training  analyst.  Four  rating 
levels  are  used.  From  an  analyst  point  of  view  the  rating  levels  are  to  be 
linked  with  customer  satisfaction: 

A.  Excellent  -  the  product  equals  or  exceeds  the  requirements  on 
every  aspect,  and  can  be  used  as  specified  -  -  the  product  is  very 
satisfactory. 

B.  Good  -  the  product  equals  or  exceeds  the  requirements  on  most 
aspects.  It  can  be  used  as  specified,  or  with  minor  limitations  -  - 
the  product  is  satisfactory. 

C.  Fair  -  the  product  doesn't  meet  the  requirements  on  some 
aspects.  It  can  however  be  used,  but  with  major  limitations  -  - 
the  product  is  almost  satisfactory. 

D.  Poor  -  the  product  does  not  meet  the  requirements.  It  cannot  be 
used  -  -  the  product  is  not  satisfactory. 


Please  turn  to  the  next  page. 
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Customer  Survey 


Characteristic 

Deeree  of  Satisfaction 

Rating 

Functionality 

Performs  user  required  functions 
with  no  limitations. 

[  ]  A:  excellent 

Performs  user  required  functions, 
with  only  minor  limitations  in 
some  of  them. 

[  ]  B:  good 

Performs  user  required  functions 
except  for  some  limitations  in 
some  of  the  functions.  These 
limitations  can  generally  be 
overcome  by  manual  procedures. 

[  ]  C:  fair 

Provided  functions  do  not  meet 
user's  requirements. 

[  ]  I>:  poor 

Briefly  describe  limications 
in  the  space  below 
if  either  of  these  blocks  are  "X” 

Comments: 
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Cust:omer  Survey  (cont) 


Characteristic 

Reliability 


Comments: 


Dep'ee  of  Satisfaction 

Performs  its  intended  functions  under 
all  foreseeable  conditions. 

Performs  its  intended  functions  under 
all  conditions.  However,  infrequent 
conditions  such  as  extreme  load,  out  of 
specification  input  data  sets  or  function 
parameter  might  result  in  incorrect  output. 

Performs  its  intended  functions.  However, 
load  conditions  and  input  data  set  and  functions 
parameters  should  be  kept  within  specification, 
so  as  to  reduce  data  loss  or  prevent 
program  interruptions. 


Rating 


[  ]  A:  excellent 


[  ]B:good 


[  ]  C:  fair 


Produces  incorrect  results,  data  losses,  and/or 

the  program  quits  even  on  correct  input  data  sets 

and  normal  load  conditions.  [  ]  i):  poor 


Briefly  describe  limitations 
in  the  space  below 
if  either  of  these  blocks  are  ”X" 


I 
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Customer  Survey  (cont) 


Usability  The  software  can  be  used  with  no  prior 

training.  On-line  itemized  "help"  is  available. 

Full  user  documentation  is  provided.  [  ]  A:  excellent 

Minimal  prior  training  is  necessary.  This 
training  can  be  done  with  a  "tutorial"  type  of 
software  package,  included  in  the  main  delivery. 

On-line  itemized  "help"  is  available.  Full  user 
documentation  is  provided.  [  ]  B;  good 

Training  is  required  prior  to  the  use  of  the 
software.  On-line  "help"  is  available,  but 
requires  prior  knowledge  of  software  to  be  used 
Documentation  requires  some  prior  knowledge 
of  software  to  be  used.  [  ]  C:  fair 

Extensive  training  is  required  before  use.  No 
documentation,  nor  help  is  available.  [  ]  D:  poor 

Briefly  describe  limitations 
in  the  space  below 
if  either  of  these  blocks  are  "X" 


Comments: 
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Customer  Survey  (cont) 


Efficiency  Response  time  remains  acceptable  under 

all  conditions. 


Response  time  remains  acceptable  under 
all  conditions,  however,  performance 
degradation  under  simulation  conditions 
is  noticeable.  The  software  indicates 
when  products  will  be  available. 


Rating 

[  ]  A;  excellent 

[  ]  B:  good 


Severe  performance  degradation  under 

simulation  conditions.  The  software 

indicates  to  the  user  requirements  to 

initiate  manual  or  alternative  procedures 

to  save  data  and  continue  work.  [  ]  C:  fair 

Unacceptable  performance  even  under 

moderate  load.  When  short  of  resources 

(file  space,  memory,  etc.)  the  software  will 

quit  with  no  prior  notice.  [  ]  0:  poor 

Briefly  describe  limitations 
in  the  space  below 
if  either  of  these  blocks  arc  "X" 

Comments : _ 
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Customer  Survey  (cont) 


Characteristic 

Deeree  of  Satisfaction 

Ratina 

Maintainability 

All  user  documentation  is  available  to 
the  user. 

[  ]  A:  excellent 

Some  documentation  is  available 
to  the  user. 

[  ] B:  good 

Documentation  is  limited  but  an  Air  Force 
point  of  contact  will  be  provided. 

[  ]  C:  fair 

No  documentation  is  provided  and  no 

Air  Force  point  of  contact  is  provided. 

I  ] D;  poor 

Briefly  describe  limitations 
in  the  space  below 
if  either  of  these  blocks  are  "X" 


Comments: 
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Cus1:omer  Survey  ( cont ) 


This  part  of  the  survey  contains  questions  about  your  requirenients  for  a 
production  version  of  the  software  product.  The  prototype  product  can  be 
used  on  computer  systems  that  meet  the  following  minimum  configuration 
requirements: 

•  80286  processor 

•  Enhanced  graphics  display 

•  Hard  disk  with  a  minimum  of  20M 
bytes  of  storage 

•  One  floppy  drive  that  can  read  and 
write  floppy  diskettes 

•  Dot  matrix  printer  capable  of  printing 
80  characters  per  line.  This  printer  must 
be  capable  of  outputting  IBM  graphics 

•  IBM  AT-compatible  keyboard 

•  DOS  3.0  or  higher 

•  640  K  bytes  of  RAM  (minimum  of  57 1  K  bytes  free) 

•  80287  coprocessor  (optional) 

1 .  Do  you  have  access  to  the  above  computer 

system,  printer,  disk  storage,  etc.?  [  ]  yes  [  ]  no 

2.  Should  the  software  provide  for  use 

of  laser  printers?  1  ]  yes  I  ]  no  (  ]  undecided 

if  "yes"  what  type  of  laser 

printer? _ 


[  ]  500 
[  ]  1,000 
[  ]  1,500 
[  ]  2,000 
[  1  2,500 
(  ]  3,000 

[  ]  3,(X)1  or  more  tasks 
[  ]  undecided 

[  ]  limit  of  300  is  acceptable 
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Customer  Survey  (cont) 


4.  I  agree  that  the  AF-MANCAP  tool  should 

included  features  for  recommending  a  new  MTBF 
when  the  operating  environment  changes  (e.g. 
when  avionics  systems  from  a  fighter  are  moved 
to  a  cargo  aircraft  the  MTBF  will  be  adjusted  from 
2,000  hours  to  8,000  hoi!rs)? 


I  ]  yes  [  ]  no  [  ]  undecided 


5.  If  an  operational  product  is  developed  would  you 
require: 

a.  Source  code  for  day-to-day  use? 

b.  Hot  line  technical  support? 


[  ]  yes  [  ]  no  [  ]  undecided 
[  ]  yes  [  ]  no  [  ]  undecided 


6.  I  could  use  the  prototype  product  to  suppon  any 
past  or  ongoing  manpower  or  personnel  analytical 
efforts? 


If  you  agree  discuss  the  applications  and  any  savings,. 


[  ]  Strongly  Agree 
[  ]  Agree 
[  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 


7.  I  agree  with  the  concept  of  developing  a  baseline 

comparison  system  from  a  large  library  that  users  can 
access? 


Comments: 


[  ]  Strongly  Agree 
[  ]  Agree 
[  i  Undecided 
[  ]  Disagree 
[  j  Strongly  Disagree 


I  _ 

{ 

I 

i 

i 
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j^ustomer  Survey  (cont) 


8.  I  agree  with  the  AF-MANCAP  simulation 
concepts? 


Comments: 


[  ]  Strongly  Agree 
[  ]  Agree 
[  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 


i 

I 

I 

s 


i 


I 


9.  I  agree  that  the  next  update  to  AF-MANCAP 
should  included  the  data  for  implementation  of 
the  Probability  of  K-Kill  computations  for  the 
components? 


Comments: 


(  J  Strongly  Agree 
[  ]  Agree 
[  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 


10. 


I  agree  that  the  next  update  to  the  AF-MANCAP 
tool  should  included  an  implementation  of 
aircraft  fuel  utilization  and  fuel  truck/operator 
suppon  requirements? 


(  ]  Strongly  Agree 
[  ]  Agree 
[  j  Undecided 
[  ]  Disagree 
[  j  Strongly  Disagree 


Comments: 


i 

i 

{ 
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Customer  Survey 


(cont) 


11.  I  agree  that  the  Terrain  substep  should  be 

retained  in  AF-MANCAP  for  potential  vehicle 
modeling  applications  in  the  Air  Force? 


Comments: 


[  ]  Strongly  Agree 
[  ]  Agree 
f  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 


12. 


I  agree  that  the  next  update  to  the  AF-MANCAP 
tool  should  include  a  rework  of  the  Fuel 
Transportation  functions  of  the  Support  Model  for 
Air  Force  applications? 


[  ]  Strongly  Agree 
[  ]  Agree 
(  ]  Undecided 
[  ]  Disagree 
[  j  Strongly  Disagree 


Comments: 


13.  I  agree  that  the  next  update  to  the  AF-MANCAP 
tool  should  include  a  rework  of  the  Ammunition 
Transportation  functions  of  the  Support  Model 
for  Air  Force  applications? 


Comments: 


[  ]  Strongly  Agree 
[  ]  Agree 
[  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 
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Customer  Survey  (cont) 


14.  I  agree  that  the  next  update  to  the 

AF-MANCAP  tool  should  implement  depot 
level  tasks  data? 


Comments; 


[  ]  Strongly  Agree 
[  ]  Agree 
[  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 


15.  I  agree  that  the  Operational  Crew  modeling 
feature  should  be  retained  in  the  AF-MANCAP 
tool? 


Comments: 


[  ]  Strongly  Agree 
[  ]  Agree 
[  ]  Undecided 
[  ]  Disagree 
[  ]  Strongly  Disagree 


16.  I  agree  that  the  AF-MANCAP  tool  should  be 
modified  for  use  with  "WINDOWS"? 


Comments: 


(  ]  Strongly  Agree 
1  ]  Agree 
[  ]  Undecided 
[  j  Disagree 
(  j  Strongly  Disagree 
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iCustomer  Survey  (cont) 


17.  Please  use  this  space  for  any  other  comments  or 

suggestions  about  the  AF-MANCAP  software  oroducts 

Comments: 


18.  Please  use  this  space  for  comments  or  suggestions  about 
the  AF-MANCAP  Overview  Briefing  and  Walk  Through 
Handbook  materials? 

Comments: 
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Additional  References : 


The  following  additional  reference  material  was  reviewed  during  the 
development  of  this  survey: 


1.  Babbie,  Earl  R.  The  Practice  of  Social  Research.  Belmont: 
Wadsworth  Publishing  Company,  Inc.  1979.  Chapter  12.  Survey  Research 
(Concept  of  Contingency  Questions,  Answer  Formats,  and  Instructions) 


2.  Boehm,  Barry  W.  Software  Risk  Management  Washington:  IEEE 
Computer  Society  Press,  1989 


3.  Shneiderman,  Ben.  Designing  the  User  Interface.  Strategies  for 
Effective  Human-Computer  Interaction.  Reading:  Addison-Wesley 
Publishing  Company,  1987. 


4.  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  Final  Report  for  Concepts  on  MPT  Estimation  (Development  of 
MANPRINT  Methods).  Alexandria,  VA,  September  1990. 
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APPENDIX  D:  SURVEY  RESULTS 


Strongly 

Strongly 

Agree 

Agree 

Undecided 

Disagree 

Disagree 

Could  Use  Prototype  Now 

All  Users 

5% 

13% 

55% 

13% 

13% 

Non-LCOM  Users 

8% 

17% 

67% 

8% 

0% 

LOOM  Users 

0% 

0% 

25% 

25% 

50% 

Need  A  Baseline  E)evelopment 

All  Users 

43% 

36% 

7% 

7% 

7% 

System 

Non-LCOM  Users 

40% 

40% 

10% 

10% 

0% 

LCOM  Users 

50% 

25% 

0% 

0% 

25% 

Agree  with  AF-MANCAP 

All  Users 

0% 

439b 

21% 

14% 

24% 

^Concepts 

Nori'LCOM  Users 

0% 

60% 

30% 

10% 

0% 

LCOM  Users 

0% 

0% 

0% 

25% 

75% 

SiKMild  Implement 

All  Users 

0% 

21% 

64% 

7% 

7% 

K'Kill  Computations 

Non-LCOM  Users 

0% 

30% 

60% 

10% 

0% 

LCOM  Users 

0% 

0% 

75% 

0<f<, 

25% 

Should  Implement 

All  Users 

7% 

29% 

43% 

14% 

7% 

Fuel  Considerations 

Non-LCOM  Users 

10% 

40% 

50% 

0% 

0% 

LCOM  Users 

0% 

0% 

25% 

50% 

25% 

Should  Retain 

All  Uscn 

0% 

36% 

29% 

29% 

7% 

Terrain  Substep 

Non-LCOM  Users 

0% 

40% 

40% 

20% 

0% 

LCOM  Users 

0% 

25% 

0% 

50% 

25% 

Should  Rework 

All  Users 

7% 

36% 

29% 

21% 

7% 

Fuel  Tnnsporulion 

Non-IX^OM  Users 

10% 

50% 

30% 

10% 

0% 

LCOM  Users 

0% 

0% 

25% 

50% 

25% 

Should  Rework 

All  Users 

7% 

36% 

50% 

0% 

7% 

Ammunition  Transportation 

Non-IX^OM  Users 

10% 

40% 

50% 

0% 

0% 

LCOM  Users 

0% 

25% 

50% 

0% 

25% 

Should  Implement 

All  Users 

21% 

54% 

0% 

7% 

7% 

Depot  Levd  Tasks 

Non-LCOM  Users 

20% 

80% 

0% 

0% 

0% 

LCOM  Users 

25% 

25% 

0% 

25% 

25% 

Should  Include 

All  Users 

0% 

43% 

36% 

14% 

7% 

C^)entional  Crew  Modeling 

Non-LCOM  Users 

0% 

50% 

40% 

10% 

0% 

LCOM  Users 

0% 

25% 

25% 

25% 

25% 

Should  Modify  to 

All  Users 

36% 

36% 

21% 

0% 

7% 

Run  Under  Windows 

Non-LCOM  Users 

40% 

40% 

20% 

0% 

0% 

LCOM  Users 

25% 

25% 

25% 

0% 

25% 
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Version! 


Nr. 


APPENDIX  E;  AF-MANCAP  SOFTWARE  DEVELOPMENT  LOG 


Product 


Pur 


1.0  Working  shell  for  upload  of 
notional  tactical  fighter 
database 


8-Jun-91 

Working  tool  for  Hay  Systems  Inc.  (HSI).  Not 
provided  to  the  AL/HRMM 

18-Jul-91 

Working  prototype  available  but  Umited  Library 
features.  Expanded  Subsystem  capability  to 
accept  SO  subsystems. 

2.0 

Working  demo  of  product 

9-Aug-91 

i 

Received  software  with  intial  Phase  I 
conversions. 

2.1  I  Expected  Value  Change 


Baseline  product  for 
demonstration  to  AFLC/ASD/ 
andHQTAC 


*  Included  new  front  screen 


•  Addition  of  glossary,  and  help  screens 


•  Notional  Fighter  Aircraft  Lib 


•  Conversion  of  MOS  to  AFSC 


•  Conversion  of  DS.GS,and  CT  to  on-equip,  off- 
equip,  and  depot 


*  Changed  mobility  equipment  group  from  miles 
to  miles/sordes 


*  Changed  the  terms  preventative  to  read 
scheduled  and  corrective  to  read  unscheduled 


12-Aug-91  Provided  Demonstration  to  AL/HRMM  and 
AF/MOR.  Also  copies  of  product  provided  to 
AL/HRMM  for  inital  testing. 


23-Aug-91  Version  2.1  received.  Not  sent  to  AL/HRMM 


•  Expenditure  rate  logic  change 


Noted  error  in  the  Support  General  and  Weapons 
Libraries.  The  wrong  group  code  (other)  was 
used.  Micro  Analysis  &  Design  (MAD)  provided 
new  library  with  mobility  group  codes. 


Version  2.2  software  received,  copied  by  HSI  and 
sent  to  AL/HRMM.  Included  Army  concurrent 
efforts: 


26-Aug 


•  Multiple  Guns 


•  Faster  HELP  screens 


•  Probablistic  Aborts 


subsystem 


•  MTTR  unit  conversions  days/hrs 


Demonstrations  at  HQ  AFLC 


Demonstrations  at  ADS/ALH 


23*SeP'91  Demonstrations  at  HQ  TAC 


104 


APPENDIX  E;  AF-MANCAP  SOFTWARE  DEVELOPMENT  LOG 


Version!  Product 


Nr.  Pur 


2.3  Off-Line  Implementation 


Ke 


Date  !  Remarks 


16-Oct-91  Version  2.3  software  received,  installed  product, 
duplicated  disks,  and  sent  HSI  copies  to 
AL/HRMM. 


•  Army  HARDMAN  III  functions  included  for 
off-line  maintenance. 


Encounter  software  error  for  simulations  of  84 
days  or  more.  Also  advised  that  AL/HRMM 
could  not  install  the  software.  Traced  problem  to 
defective  disk  duplication  by  HSI. 


New  disks  received  firom  MAD.  HSI  found  that 
working  files  could  not  be  deleted. 


Replacement  disks  received,  installed  and  tested 
on  HSI  computo'.  Sent  MAD  original  disks  to 
AL/HRMM.  Also,  included  correction  of  the 
following  defects  noted  during  HSI  testing; 


*  Fixed  simulation  limit  of  68  days.  Goal  of  180 
days  established  for  MANCAP. 


*  Added  form  feed  at  the  end  of  the  manpower 
analysis  print  routine. 


•  Revised  the  "Identify  Maintenance  Manpower 
Resources"  edit  routines. 


•  Software  changed  to  allow  deletion  of  woricing 
files. 


HSI  noted  that  product  aborted  the  180  day 
simulation  after  88.7%  and  a  run  time  of  1342 
minutes. 


Version  2.4  software  ready  to  ship  to  AL/HRMM 
Advised  AL/HRMM  that  we  would  not  ship  the 
product  Army  off-line  features  did  not  work. 
Only  Remove  &  Replace  tasks  counted  for  off¬ 
line 


HSI  conducted  testing  to  verify  that  simulation 
run  times  of  180  days  could  be  finished. 


Version  2.4  software  shipped  to  AL/HRMM. 
Disks  were  installed  and  tested  and  installed  by 
HSI. 


•  Included  modification  of  the  off-line  to 
consido:  troubleshoot  remove  &  replace,  inspec 
adjust/repair,  and  test/check. 


*  Included  fix  to  the  reduce  simulation  run  time 
by  half. 
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5-Dec-91 

Notifed  by  AL/HRMM  that  front  screen  was 
missing.  Also,  could  not  complete  simulations 
for  30  days.  Suspected  defective  disk. 

2.5 

Off-line  Air  Force 

r  6-Dec-91 

Replacement  disks  provided  to  AL/HRMM  direct 
from  MAD. 

•  Corrected  version  numbering. 


*  Repackaged  the  front  screen. 


ll-Dec-91 

AL/HRMM  indicated  that  product  would  not 
icomplete  a  30  day  simulation.  Tried  installauc.'. 
on  two  386  computers  and  one  286  computer. 

ll-Dec-91 

Successful  simulations  initiated  by  HSI  and  MAD 
from  duplicate  software  on  WYSE  computer. 

Also  front  screen  was  missing. 

12-Dec-91 

Notified  by  AL/HRMM  that  front  screen  was 
missing. 

17.Dec.91 


18-Dec-91 


20-Dec-91 


vided  to  AL/HRMM  direct 


AL/HRMM  found  error  in  the  total  entry  for 
System  Availability  Report 


HSI  &  MAD  unable  to  duplicate  the  error. 
Requested  AL/HRMM  download  the  model  and 
send  disk  to  HSI  for  analysis. 


